-- Logs begin at Wed 2022-07-13 19:58:21 UTC, end at Wed 2022-07-13 21:29:55 UTC. -- Jul 13 20:50:33.716985 dv-cinder-799871-27-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 13 20:50:33.724339 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 13 20:50:33 2022] *** Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: nodename: dv-cinder-799871-27-iscsi Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: machine: x86_64 Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: clock source: unix Jul 13 20:50:33.726340 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: pcre jit disabled Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: detected number of CPU cores: 4 Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: current working directory: / Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: detected binary path: /usr/bin/uwsgi-core Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: your processes number limit is 47680 Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: your memory page size is 4096 bytes Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: detected max file descriptor number: 2048 Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: lock engine: pthread robust mutexes Jul 13 20:50:33.726810 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: thunder lock: enabled Jul 13 20:50:33.728957 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 13 20:50:33.728957 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 13 20:50:33.770847 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: Python main interpreter initialized at 0x5632c6caa9c0 Jul 13 20:50:33.770847 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: python threads support enabled Jul 13 20:50:33.770847 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: your server socket listen backlog is limited to 100 connections Jul 13 20:50:33.770847 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: your mercy for graceful operations on workers is 90 seconds Jul 13 20:50:33.771187 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: mapped 403077 bytes (393 KB) for 2 cores Jul 13 20:50:33.771262 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: *** Operational MODE: preforking *** Jul 13 20:50:33.771536 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: *** uWSGI is running in multiple interpreter mode *** Jul 13 20:50:33.771536 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: spawned uWSGI master process (pid: 80739) Jul 13 20:50:33.771736 dv-cinder-799871-27-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 13 20:50:33.774652 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: spawned uWSGI worker 1 (pid: 80748, cores: 1) Jul 13 20:50:33.774652 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: spawned uWSGI worker 2 (pid: 80749, cores: 1) Jul 13 20:50:33.774652 dv-cinder-799871-27-iscsi devstack@keystone.service[80739]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 13 20:50:39.174978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:50:39.174978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: from cryptography.utils import int_from_bytes Jul 13 20:50:39.175766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:50:39.175766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: from cryptography.utils import int_from_bytes Jul 13 20:50:39.176192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:50:39.176192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: from cryptography.utils import int_from_bytes Jul 13 20:50:39.176639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 20:50:39.176639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: from cryptography.utils import int_from_bytes Jul 13 20:50:39.223806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 20:50:39.223995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 20:50:39.224185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] command line args: [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 20:50:39.224452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 20:50:39.224629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ================================================================================ {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 20:50:39.224738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 20:50:39.224880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] admin_token = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.224965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 20:50:39.225112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] command line args: [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 20:50:39.225194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] config_dir = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.225299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.225381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 20:50:39.225483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] config_source = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.225560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ================================================================================ {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 20:50:39.225662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] debug = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.225781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] admin_token = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.225865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] config_dir = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] config_source = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] debug = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.226972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_config_append = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_dir = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.227916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_file = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_config_append = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_dir = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.228827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.229538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_file = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.229660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.229786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.229908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.230997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.231893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] publish_errors = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232748 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.232972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] publish_errors = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] standard_threads = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.233941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] standard_threads = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] use_journal = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] use_json = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.234951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] use_stderr = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] use_syslog = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] use_journal = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] use_json = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.235858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.235971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] use_stderr = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.236083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.236201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] use_syslog = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.236201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 20:50:39.236433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.236547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.236660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.236772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.236891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.236891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.237931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.238898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.239827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.240930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.241960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.242924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.243967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.244893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.245887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.246924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.247937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.248936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.249886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.250953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.251962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.252993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.253907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.254959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.255992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.256987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.257956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.258951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.259777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.external = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.260902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.external = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.password = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] auth.token = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.261974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.password = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] auth.token = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.262955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.263893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.264008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.264123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.264238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.264365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.264482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.264596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.265907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.266957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.267966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.268950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.269922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.270895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.271953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.272923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.273999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.274999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.275126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.275244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.275357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.275471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.275585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.276192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.276321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.276438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.276552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.276666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.276781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.277433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.277613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.277734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.277850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.277966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.278928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.279981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.280914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.281977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.282980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.283939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.284990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.285934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.286897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.287956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.288900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.289956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.290072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.290196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.290312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.290428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.290543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.290935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.291050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.291178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.291302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292628 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.292975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.293919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] role.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.294965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] role.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] role.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.295899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] role.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.296962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.297904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.298989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.299926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.300976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.301912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.302898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.303976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.304908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.305969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.306948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.connection = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.307920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.connection = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.308987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.309930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.310932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.311794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80749) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 20:50:39.312021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.312021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.312021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.312021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 20:50:39.312021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80748) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 20:50:39.341057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.341282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.341509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.341595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.341817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.342049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.349909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.350390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.378344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.378575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.378684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.378846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.522037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.522212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.570931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.571148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.571257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.571384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.635756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.635980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.636979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.637067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.661766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.661882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.661976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.662964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.663037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.663109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.663197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.674327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.674516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.687938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.688022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.688150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.688232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.688364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.688443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80749) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.688579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80748) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 20:50:39.705907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.706114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:50:39.706391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.706602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:50:39.706780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.706900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:50:39.707169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.707248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.707379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 20:50:39.707461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.707655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.707777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.707906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.707987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.708157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.708407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.719459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.719733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.719851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.719999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.720113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.720278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.720393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.720582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.728590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.728886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.731952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.732229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.732310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.732505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.732586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.732783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.732867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.733933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.734925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.762071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.762194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.763717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.763846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.763995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.764127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.769301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.769445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.769579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.769721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.769856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.769997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.777959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.778118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.778210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.778398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.782710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.782959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.783041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.783241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.783335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.783519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.783599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.783791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.783872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.784849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.803655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.803925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.804007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.804197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.809020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.809317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.812709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.813002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.813081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.813291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.818206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.818485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.821857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.822145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.826095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.826388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.826469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.826708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.826790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.827840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.846854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.847110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.847191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.847387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.847480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.847649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.847730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.847919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.855658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.855963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.857369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.857641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.857732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.857930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.860870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.861149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.861231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.861429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.861512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.861706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.861797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.861988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.862069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.862332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.875898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.876172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.876255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.876446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.876537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.876729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.876819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.877825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.897411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.897704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.900764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.901048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.904145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.904425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.905745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.906031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.907549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.907808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.907890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.908094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.908176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.908381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.915502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.916008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.917661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.918640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.920936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.921224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.921505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.921781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.921911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.922201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.922482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.922787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.931156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.931442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.931718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.932674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.932955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.933231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.940196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80748) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.940476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.940754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.941026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.941225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80749) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 20:50:39.941307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.941500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.941582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.941776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.941859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.942883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80748) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.943840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.944037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.944299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.944561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.944831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80749) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 20:50:39.982508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5632c6caa9c0 pid: 80748 (default app) Jul 13 20:50:39.983542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5632c6caa9c0 pid: 80749 (default app) Jul 13 20:50:39.985863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7a14-a7fb-4acd-a497-705d41357d3c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:39.986159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7a14-a7fb-4acd-a497-705d41357d3c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:39.986441 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce7a14-a7fb-4acd-a497-705d41357d3c None None] PATH_INFO: `/v3` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:39.990588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Wed Jul 13 20:50:39 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:47.469766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:47.470323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:47.470323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:47.474718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:50:47.480119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:47.480536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:47.480678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:47.549917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: INFO dbcounter [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] Registered counter for database keystone Jul 13 20:50:47.571187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG oslo_db.sqlalchemy.engines [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80748) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 20:50:47.577865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writer thread running {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 20:50:47.752179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG passlib.handlers.bcrypt [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80748) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 20:50:47.752604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG passlib.handlers.bcrypt [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80748) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 20:50:47.806465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:47.808608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:47.928232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 13 20:50:48.122588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:48.192943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: INFO dbcounter [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] Registered counter for database keystone Jul 13 20:50:48.212013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG oslo_db.sqlalchemy.engines [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80749) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 20:50:48.218586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writer thread running {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 20:50:48.335669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e79b4-5a93-4f03-a266-8a7c9e70cedd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:48.340497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:48.341158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:48.341445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:48.341726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:48.349729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:48.401674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 20:50:48.401964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79828d8-bdb4-497e-86c1-32fab2488d44 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:48.411404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Wed Jul 13 20:50:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 20:50:48.418182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:48.432492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009807c2-2563-4b24-b76b-d8f21e411003 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:48.436826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:48.437947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:48.438239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:48.438519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] PATH_INFO: `/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:48.442585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] RBAC: Authorizing `identity:get_project(project_id=c113e9e1592248a9b12fd4d2b385aa5a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:48.493671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 20:50:48.494212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b82e3e-f121-494a-a5d8-b7cdc5e04f02 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:48.498537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Wed Jul 13 20:50:48 2022] GET /identity/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:50.474923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67ee0c-7969-4687-884e-214dee615522 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:50.475435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67ee0c-7969-4687-884e-214dee615522 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:50.475519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67ee0c-7969-4687-884e-214dee615522 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:50.479612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:50:50.485025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:50.485393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:50.485588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:50.504530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:50.506053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:50.514483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 20:50:50.536074 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67ee0c-7969-4687-884e-214dee615522 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:50.548073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67ee0c-7969-4687-884e-214dee615522 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:50.552249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dcb4d-dea4-4362-b7c1-423e78d85a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:50.552872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dcb4d-dea4-4362-b7c1-423e78d85a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:50.553157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dcb4d-dea4-4362-b7c1-423e78d85a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:50.553433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dcb4d-dea4-4362-b7c1-423e78d85a3d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:50.561537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dcb4d-dea4-4362-b7c1-423e78d85a3d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:50.563342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dcb4d-dea4-4362-b7c1-423e78d85a3d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:50.571969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Wed Jul 13 20:50:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 20:50:50.577543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:50.586349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45d623-9587-446d-a28a-9e4a6fb64aed None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:50.590743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f94e53-2550-4ae8-b565-77a1e3bbd63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:50.591596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f94e53-2550-4ae8-b565-77a1e3bbd63a admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:50.591884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f94e53-2550-4ae8-b565-77a1e3bbd63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:50.592162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f94e53-2550-4ae8-b565-77a1e3bbd63a admin admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:50.593638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f94e53-2550-4ae8-b565-77a1e3bbd63a admin admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:50.595508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f94e53-2550-4ae8-b565-77a1e3bbd63a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:50.597083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Wed Jul 13 20:50:50 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:52.642108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedbb49-8ac4-472c-9e8d-0fd37f8253e3 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:52.642677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedbb49-8ac4-472c-9e8d-0fd37f8253e3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:52.642677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedbb49-8ac4-472c-9e8d-0fd37f8253e3 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:52.644065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:50:52.649246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:52.649720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:52.649856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:52.666387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:52.667590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:52.711630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0920d-a99d-47cc-9778-6e478ba85141 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:52.711917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0920d-a99d-47cc-9778-6e478ba85141 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:52.712200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0920d-a99d-47cc-9778-6e478ba85141 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:52.713567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:50:52.718472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:52.718791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:52.719075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:52.751226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:50:52.758090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG passlib.handlers.bcrypt [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80749) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 20:50:52.758512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG passlib.handlers.bcrypt [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80749) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 20:50:52.774359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:52.797496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:52.804380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:52.806003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:52.881410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bafa6-5254-4156-8d20-7f70abb7f83e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:52.905775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:50:52.914246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:52.915074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:52.915615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:52.916411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:52.927649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:50:52.928127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:50:52.939581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:50:52.940492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:50:52.941421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:50:52.946544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:53.030047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:53.033015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df31c4e-7350-4858-9a26-65d5f69c2252 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.035411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:53.050497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cf2bf-97b5-402d-b628-5e2bfddc2cb5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.072076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.073011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.073340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.073613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.084433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.085127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.085422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.085704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.086471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:50:53.086814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.088255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.099989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:50:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:53.105918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:53.140199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.141857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.203468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9354f-3360-4d25-839a-705db9c57310 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.236960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.237832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.238118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.238199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 8/16] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:50:53.238408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.239640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.241099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.246648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:50:53 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:53.247771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:53.247879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:53.250652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:53.262774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:53.347785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415c1f-5089-4306-83e5-c63120f29ff6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.351171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d6aa2-e46a-4808-8f6d-92f889543027 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.383835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.384675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.385445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.385722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.386643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.386924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.387442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.387726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.387998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.388508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.390289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 10/18] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:50:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:53.395706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:53.406480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.407991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:50:53.428520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:50:53.429339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:50:53.430112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:50:53.430112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:50:53.430112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:50:53.430112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:50:53.430112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 9/19] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:50:53.436063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:53.477647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:53.495534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:53.499037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c13d23-eee8-460e-bd34-0cc15378d40e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.532346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.533006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.533296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.533577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.534421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.domain.id=default, user.id=a762106a78c649808cb7ea162b5997db) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:50:53.534768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.536215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fd569-1604-4f72-9579-3e4fe8488054 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.539128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574cb18-31b9-4fcd-90bd-3fddf2d35739 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:53.543747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 11/20] 10.222.0.39 () {56 vars in 1322 bytes} [Wed Jul 13 20:50:53 2022] GET /identity/v3/role_assignments?user.id=a762106a78c649808cb7ea162b5997db&scope.domain.id=default&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:53.575810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:53.576491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:53.576774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:53.577047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:53.577798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:50:53.578107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:53.579586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ace8e-170e-4f53-ae23-ab376d5b944b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:53.599706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 10/21] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:55.907672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81803f5d-18d3-48c1-8d3d-dd260a9b8124 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:55.908181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81803f5d-18d3-48c1-8d3d-dd260a9b8124 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:55.908253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81803f5d-18d3-48c1-8d3d-dd260a9b8124 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:55.909780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 12/22] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:50:55.924243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:55.924711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:55.924781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:55.952450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:55.953650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:55.984134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7c6e2-57f7-4229-8378-45efaffaf7a2 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:55.984474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7c6e2-57f7-4229-8378-45efaffaf7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:55.984705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7c6e2-57f7-4229-8378-45efaffaf7a2 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:55.986285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 13/23] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:50:55.991421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:55.991709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:55.991986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.011553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:56.012736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:56.071132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 11/24] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:50:56.109686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.148214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 14/25] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:50:56.198305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:56.211978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.304792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d590a7a-0af6-4bb6-8a85-859b47ba2410 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:56.351804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:56.353033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.354068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.354548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.379237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:50:56.389714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:50:56.390552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:50:56.391356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:50:56.394120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 12/26] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:50:56.399867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.439457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:56.488293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee75c3b-5c70-4312-8da4-61a6df06d377 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:56.525903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623537e-f2a9-4a54-9529-bbe136d2e6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:56.526992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623537e-f2a9-4a54-9529-bbe136d2e6a3 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.527503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623537e-f2a9-4a54-9529-bbe136d2e6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.528026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623537e-f2a9-4a54-9529-bbe136d2e6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.529163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623537e-f2a9-4a54-9529-bbe136d2e6a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:56.530836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623537e-f2a9-4a54-9529-bbe136d2e6a3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:56.565436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82481a5f-c62c-4033-9902-77b797a5c750 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:56.574386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:56.619451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:56.620160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.620446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.620727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.621516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:50:56.621822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:56.623264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025f0a0-5c8e-4ec8-976f-2cb43a7634b4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:56.636065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 13/27] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:56.637590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73b06-6b55-44d1-a0d5-b94841bc5226 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.637887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73b06-6b55-44d1-a0d5-b94841bc5226 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.638164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc73b06-6b55-44d1-a0d5-b94841bc5226 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.639408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 14/28] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:50:56.643455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f651274-45a9-4bae-93f7-a313490594c0 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.643742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f651274-45a9-4bae-93f7-a313490594c0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.644025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f651274-45a9-4bae-93f7-a313490594c0 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.647262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 15/29] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:50:56.648699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f651274-45a9-4bae-93f7-a313490594c0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.710545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 15/30] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Jul 13 20:50:56 2022] POST /identity/v3/roles => generated 219 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:50:56.712416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25667f-34be-4ac5-bda4-4c1d78372783 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.713324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25667f-34be-4ac5-bda4-4c1d78372783 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.713751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25667f-34be-4ac5-bda4-4c1d78372783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.732480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-eb25667f-34be-4ac5-bda4-4c1d78372783 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:56.734189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-eb25667f-34be-4ac5-bda4-4c1d78372783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:56.772011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f651274-45a9-4bae-93f7-a313490594c0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:56.814652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 16/31] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:50:56.814652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.814652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.814652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.826909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:56.828033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:56.828483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:56.828906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:56.830643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:56.831194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:56.831999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:56.833929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:56.837217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 16/32] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:50:56 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:56.842930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.889581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:56.909526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:56.920435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 17/33] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:50:56.921346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:56.975290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3a4b9-bc7a-4f26-9681-20617baca0ba None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.022728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.024435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.025795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.026287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.027752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.030242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.032659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 17/34] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:50:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:50:57.034770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:57.075707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed19e22-fc0f-4c3e-b18c-d06bebea4498 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.111463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.112212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.112546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.112857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.113588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.115168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.130495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 18/35] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Jul 13 20:50:56 2022] POST /identity/v3/roles => generated 225 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:50:57.132755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:57.175874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2abd0-6b87-475f-b618-043078a64d93 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.179138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:57.201665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:57.221116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.222306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.222863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.223364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.236123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.237999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.255033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c0930-ed61-4442-ab06-7978dcfc9c31 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:50:57.258658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:50:57.259552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:50:57.260831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:50:57.260831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:50:57.260831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:50:57.260831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:50:57.262795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 18/36] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:50:57.269235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:57.295900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.297249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.297542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.297880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] PATH_INFO: `/v3/domains/default/users/a762106a78c649808cb7ea162b5997db/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.300379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a762106a78c649808cb7ea162b5997db, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.302239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.325081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 19/37] 10.222.0.39 () {58 vars in 1361 bytes} [Wed Jul 13 20:50:57 2022] PUT /identity/v3/domains/default/users/a762106a78c649808cb7ea162b5997db/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:50:57.397335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abfdee-966d-445f-9ecf-66040f9beff6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.436025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.437057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.437567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.438038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.439012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:50:57.439558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.441225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02899e5-f9c9-4757-b7b5-22758390c030 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.455266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 19/38] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:57.463358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:57.597814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5320-1bb2-4b37-8647-a0a5e9a43997 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.613464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:57.637532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.638211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.638503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.638858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.639579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.641080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.652842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27d8e0-5a6f-4e88-9694-44c8be32d519 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.653136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27d8e0-5a6f-4e88-9694-44c8be32d519 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.653420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27d8e0-5a6f-4e88-9694-44c8be32d519 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.654837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 20/39] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:50:57.659902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569cfa8-5ae7-4e46-bb2b-6664c31f111b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.660220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569cfa8-5ae7-4e46-bb2b-6664c31f111b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.660479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569cfa8-5ae7-4e46-bb2b-6664c31f111b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.672417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 20/40] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:50:57 2022] POST /identity/v3/projects => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:50:57.680014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1569cfa8-5ae7-4e46-bb2b-6664c31f111b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:57.681262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1569cfa8-5ae7-4e46-bb2b-6664c31f111b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:57.703058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:57.761493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 21/41] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:50:57.787136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:57.877395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e99cf-e83d-4e36-8e43-c4cbbefba49d None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.877690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e99cf-e83d-4e36-8e43-c4cbbefba49d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.878115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e99cf-e83d-4e36-8e43-c4cbbefba49d None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.879686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 22/42] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:50:57.884720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.885283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.885771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.907472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:57.908424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91c5c-15a9-4f99-9881-7e08d16d5010 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:57.908980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:57.945415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:57.946218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:57.946505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:57.946816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:57.948017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:57.949474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:57.951383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 21/43] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:50:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:57.956437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:57.988356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 23/44] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:50:57.997529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:58.013896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:58.070948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbfa2a-bdc7-4b86-9cb3-311eaa9ec746 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:58.092716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:58.119777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9320ad-bbf5-4db9-8598-0033ee156186 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:58.157808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a70e9-4e5d-443a-a22a-f8c12cd78119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:58.158412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a70e9-4e5d-443a-a22a-f8c12cd78119 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:58.158723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a70e9-4e5d-443a-a22a-f8c12cd78119 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:58.159000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a70e9-4e5d-443a-a22a-f8c12cd78119 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:58.159767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a70e9-4e5d-443a-a22a-f8c12cd78119 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:58.161190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a70e9-4e5d-443a-a22a-f8c12cd78119 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:58.214794 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88abd3fa-7549-4703-9d99-13716c13c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:58.215464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abd3fa-7549-4703-9d99-13716c13c322 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:58.215752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abd3fa-7549-4703-9d99-13716c13c322 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:58.216034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88abd3fa-7549-4703-9d99-13716c13c322 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:58.216754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abd3fa-7549-4703-9d99-13716c13c322 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:58.218211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88abd3fa-7549-4703-9d99-13716c13c322 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:58.244284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 22/45] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:50:57 2022] POST /identity/v3/projects => generated 312 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:50:58.391418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 24/46] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Jul 13 20:50:58 2022] POST /identity/v3/roles => generated 223 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:50:59.486507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151f0c-1da2-469c-87e7-97c0ca2add1c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:59.487031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151f0c-1da2-469c-87e7-97c0ca2add1c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:59.487109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a151f0c-1da2-469c-87e7-97c0ca2add1c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:59.488476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 23/47] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:50:59.494495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:59.494923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:59.495168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:59.513197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:50:59.514737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:50:59.579270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 25/48] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:50:59.602190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151f0c-1da2-469c-87e7-97c0ca2add1c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:59.685418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:59.701971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a151f0c-1da2-469c-87e7-97c0ca2add1c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:59.734468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:59.735284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:59.735574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:59.735857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:59.746126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:50:59.753696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:50:59.754600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:50:59.755446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 24/49] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:50:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:50:59.761266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:50:59.825007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:50:59.860620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b4de0-f9fc-48e0-83e1-05b5031eed35 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:50:59.893659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:50:59.894327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:50:59.894610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:50:59.894913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:50:59.895663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:50:59.895969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:50:59.897473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:50:59.906114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 26/50] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:50:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:50:59.911623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:00.008924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d862e3e-f382-4257-8646-0807d3356be2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:00.042154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:00.043056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.043340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.043612 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.044805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:00.046259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:00.047919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 25/51] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:50:59 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:00.053397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:00.149670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683dfdf-d321-42f1-846b-51a4498c7395 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:00.175644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:00.184199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab645e87-8f67-44a7-b9b7-58af30095c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:00.184996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab645e87-8f67-44a7-b9b7-58af30095c10 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.185279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab645e87-8f67-44a7-b9b7-58af30095c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.185554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab645e87-8f67-44a7-b9b7-58af30095c10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.186761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab645e87-8f67-44a7-b9b7-58af30095c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:00.188260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab645e87-8f67-44a7-b9b7-58af30095c10 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:00.189856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 27/52] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:00.195541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:00.212027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:00.280559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7acae-d443-41b4-945f-a186716c6273 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.280848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7acae-d443-41b4-945f-a186716c6273 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.281125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7acae-d443-41b4-945f-a186716c6273 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.282508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 28/53] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:00.287401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.287693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.287976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.300050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3b650-8eba-44cf-bb37-4599ad3f8c5c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:00.304204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:00.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:00.333870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:00.334526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.334834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.335110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.335948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.domain.id=default, user.id=a762106a78c649808cb7ea162b5997db) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:00.336260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:00.337731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:00.343459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 26/54] 10.222.0.39 () {56 vars in 1322 bytes} [Wed Jul 13 20:51:00 2022] GET /identity/v3/role_assignments?user.id=a762106a78c649808cb7ea162b5997db&scope.domain.id=default&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:00.373770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 29/55] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:00.397603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:00.479872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:00.502636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5911db-6c47-4a45-aa91-95590d4117ec None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:00.535757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c4f7d1-f255-40a1-94c6-ea3af348b88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:00.536801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4f7d1-f255-40a1-94c6-ea3af348b88d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.537086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4f7d1-f255-40a1-94c6-ea3af348b88d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.537358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c4f7d1-f255-40a1-94c6-ea3af348b88d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.538545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4f7d1-f255-40a1-94c6-ea3af348b88d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:00.540054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c4f7d1-f255-40a1-94c6-ea3af348b88d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:00.542578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 27/56] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:00.548112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:00.645014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4abbe4-6a9a-4313-bf15-e8128619d95e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:00.669977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:00.678497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d484fd-8cc6-4c86-b301-6c95e3059b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:00.679187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d484fd-8cc6-4c86-b301-6c95e3059b31 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:00.679475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d484fd-8cc6-4c86-b301-6c95e3059b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:00.679754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d484fd-8cc6-4c86-b301-6c95e3059b31 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:00.680473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d484fd-8cc6-4c86-b301-6c95e3059b31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:00.682475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d484fd-8cc6-4c86-b301-6c95e3059b31 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:00.707712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 30/57] 10.222.0.39 () {60 vars in 1140 bytes} [Wed Jul 13 20:51:00 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:02.702772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ad6df-6697-492e-b1f8-906a251a7470 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:02.703288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ad6df-6697-492e-b1f8-906a251a7470 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:02.703381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ad6df-6697-492e-b1f8-906a251a7470 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:02.704696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 28/58] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:02.709694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:02.710065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:02.710257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:02.727322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:02.728527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:02.792481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 31/59] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:02.816076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ad6df-6697-492e-b1f8-906a251a7470 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:02.913491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ad6df-6697-492e-b1f8-906a251a7470 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:02.944785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad2d32-723a-4a18-8600-cf9eb7932fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:02.945554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2d32-723a-4a18-8600-cf9eb7932fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:02.945838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2d32-723a-4a18-8600-cf9eb7932fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:02.946115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad2d32-723a-4a18-8600-cf9eb7932fa2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:02.947821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad2d32-723a-4a18-8600-cf9eb7932fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:02.949259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad2d32-723a-4a18-8600-cf9eb7932fa2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:02.950856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 29/60] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:02.955997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:03.052260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e954b2e-f80c-44f7-b994-d7a6ad600c6a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:03.061587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:03.086923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dc391-59b5-4866-9c5f-e60319cf709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:03.087580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc391-59b5-4866-9c5f-e60319cf709f None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:03.087865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc391-59b5-4866-9c5f-e60319cf709f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:03.088141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc391-59b5-4866-9c5f-e60319cf709f None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:03.088893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc391-59b5-4866-9c5f-e60319cf709f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:03.090335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc391-59b5-4866-9c5f-e60319cf709f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:03.114106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 32/61] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:02 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:06.154510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9514fe-826c-4206-9f55-360a550c1abd None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.155564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9514fe-826c-4206-9f55-360a550c1abd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.156082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9514fe-826c-4206-9f55-360a550c1abd None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.157765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 30/62] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:06.163154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.163958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.164440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.170146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffb3fe-d014-4c17-8fc8-68f007da65ae None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.170847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffb3fe-d014-4c17-8fc8-68f007da65ae None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.171370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffb3fe-d014-4c17-8fc8-68f007da65ae None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.172952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 31/63] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:06.177740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.178275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.178849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.195610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:06.197391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:06.209864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:06.211072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:06.221352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:06.300692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 32/64] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:06.322298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 33/65] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:06.335561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.355693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.468079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43aedc-4aa8-41f1-85bd-f5b0ca6d2b5b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:06.469747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e6dfa-4494-46f8-8d04-717cd92597a5 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:06.471018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:06.505418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188de24b-73bc-4bcb-a732-2c7154c50904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:06.506224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188de24b-73bc-4bcb-a732-2c7154c50904 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.506519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188de24b-73bc-4bcb-a732-2c7154c50904 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.506858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188de24b-73bc-4bcb-a732-2c7154c50904 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.507288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:06.508319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.509333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.509333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.518106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188de24b-73bc-4bcb-a732-2c7154c50904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:06.519728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:06.527526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:06.528558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:06.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-188de24b-73bc-4bcb-a732-2c7154c50904 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:06.531404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:06.532397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:06.533387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:06.538187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 33/66] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:06.538318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 34/67] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:06.538405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa2f11-88af-4d5a-8d72-6d3e29e17009 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.538485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.538707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa2f11-88af-4d5a-8d72-6d3e29e17009 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.538707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa2f11-88af-4d5a-8d72-6d3e29e17009 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.538707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 34/68] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:06.542999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa2f11-88af-4d5a-8d72-6d3e29e17009 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.548329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:06.644983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d922e-61aa-4c6e-b31a-c58afa7e0568 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:06.677285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa2f11-88af-4d5a-8d72-6d3e29e17009 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:06.682637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:06.683320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.683604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.683883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.684648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:06.684952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:06.686450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3520a-9cbd-44a2-9e5f-fbf674d94383 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:06.700729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 35/69] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:06.702091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.702386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.702819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.716704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:06.717658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.717951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.718291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.719054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:06.719371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:06.720830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:06.721164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:06.722470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:06.731480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 35/70] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:06.732298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.738198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:06.818649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 36/71] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:06.818649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.818925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:06.851046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc7f25-8032-4fba-850e-9d75eb5e2f9e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:06.888740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f302597-e76b-4690-8391-45c24aaff1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:06.889878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f302597-e76b-4690-8391-45c24aaff1dc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.890163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f302597-e76b-4690-8391-45c24aaff1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.890439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f302597-e76b-4690-8391-45c24aaff1dc None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.906798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f302597-e76b-4690-8391-45c24aaff1dc None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:06.908315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f302597-e76b-4690-8391-45c24aaff1dc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:06.914201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 36/72] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:06.914201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755667-557b-455a-83b2-7a0bf122b052 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.914201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755667-557b-455a-83b2-7a0bf122b052 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.914201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29755667-557b-455a-83b2-7a0bf122b052 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.914201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 37/73] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:06.914201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755667-557b-455a-83b2-7a0bf122b052 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.935857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7db21-c4bb-48a0-9137-d027ecce023d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:06.973187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:06.973995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:06.974282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:06.974561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:06.975943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:06.977272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:06.979144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 37/74] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:06.979836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:06.997563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.024210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29755667-557b-455a-83b2-7a0bf122b052 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.048722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.060544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27feb0ee-0cef-4953-a96e-f68c5ff708f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.061352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feb0ee-0cef-4953-a96e-f68c5ff708f3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.061644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feb0ee-0cef-4953-a96e-f68c5ff708f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.061929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27feb0ee-0cef-4953-a96e-f68c5ff708f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.072731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27feb0ee-0cef-4953-a96e-f68c5ff708f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-27feb0ee-0cef-4953-a96e-f68c5ff708f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:07.080731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:07.081631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:07.082367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 38/75] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:07.083396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a6771-7367-49c5-955f-b5a7ee724d3a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.083491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.083491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.083736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.100358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:07.101563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:07.116222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7622a39-ea6c-4539-8c4f-fd671e52c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.116876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7622a39-ea6c-4539-8c4f-fd671e52c138 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.117163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7622a39-ea6c-4539-8c4f-fd671e52c138 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.117443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7622a39-ea6c-4539-8c4f-fd671e52c138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.167085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 39/76] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:07.167908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.195619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7622a39-ea6c-4539-8c4f-fd671e52c138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.197129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7622a39-ea6c-4539-8c4f-fd671e52c138 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.211582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.264349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e1530-2f9b-428f-bd52-d9e23f74a593 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.280402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.302116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 38/77] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:07.302116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5f2d9-01a5-430e-9a51-d4205b4ecffb None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.302116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5f2d9-01a5-430e-9a51-d4205b4ecffb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.302116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5f2d9-01a5-430e-9a51-d4205b4ecffb None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.302116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 39/78] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:07.302116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5f2d9-01a5-430e-9a51-d4205b4ecffb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.305727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.307199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.307591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.307980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.375072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.376540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.403166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5f2d9-01a5-430e-9a51-d4205b4ecffb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.417305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.438147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.438839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.439128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.439410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.440165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:07.440481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.441858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.449575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 40/79] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:07.450354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.477058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 40/80] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:07.478444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.492531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.560664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24a76e-f48d-4585-b74d-3524e5c17c6f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.576491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361846-01ad-44a3-8d91-1951a6497bca None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.593491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad5b48-1646-416f-a558-4ccec86b8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.594288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad5b48-1646-416f-a558-4ccec86b8b49 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.594577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad5b48-1646-416f-a558-4ccec86b8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.594879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad5b48-1646-416f-a558-4ccec86b8b49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.605416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad5b48-1646-416f-a558-4ccec86b8b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:07.610118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.610992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.611280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.611559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.612738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0dad5b48-1646-416f-a558-4ccec86b8b49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:07.613192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:07.614070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:07.619907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:07.620631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.620710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 41/81] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:07.620710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.620710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.620914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 41/82] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:07.620991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.621066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.635876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:07.637095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:07.667296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.700012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.703730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 42/83] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:07.704631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.721761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f66dc3-d7d6-4461-af7f-5094f45d6cc5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.755272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.756143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.756427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.756700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.757897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.759370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.762409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 42/84] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:07.762409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.804556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b98ff-3092-4f9c-88b1-468b65684b09 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.836722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.838029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.838314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.838592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.839818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.841283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.843039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 43/85] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:07.843795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.861454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c046159-cfeb-42d8-9aa0-283d2c41f14e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.876500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.896016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.896676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.896959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.897236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.897980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:07.898287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.899720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.907487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 43/86] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:07.908264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:07.924402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:07.943047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9ec2-b389-4f22-bbca-c8a622c20e47 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:07.976527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:07.977187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:07.977476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:07.977753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:07.979119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=97ea2dd3c47c492b95e460c97de31345) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:07.979437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:07.980887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:07.989267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 44/87] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/role_assignments?user.id=97ea2dd3c47c492b95e460c97de31345&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=8182238074534b76a65d54c2edbeffad => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:07.990060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.019268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bfba1-f11a-459a-b9dd-4c606c718087 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.058860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.059672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.059963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.060244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.061500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.063115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.069310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 44/88] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:08.069310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.115960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:08.121709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e3069-3ac3-48ea-8436-729230027252 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.154482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:08.163887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.164579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.164870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.165891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.179997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90608974-cf19-4b6b-bc77-33730edca6e3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.222303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.223000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.223433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.223725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.225166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=a762106a78c649808cb7ea162b5997db) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:08.225490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.227689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.233636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.242698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 45/89] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:08 2022] GET /identity/v3/role_assignments?user.id=a762106a78c649808cb7ea162b5997db&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:08.243000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.243092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.370152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02ac85-56c5-416a-842d-d69752d92ec6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.387287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:08.394578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 45/90] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:08.395287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.409502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.410711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.411180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.411462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.412710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.414170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.421052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 46/91] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:08 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:08.421052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.436178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:08.524609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11479e64-064a-43a6-a314-b95a70dd3503 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.527811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522d79c-e78c-49c8-be52-edfd6db154b6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.566664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.567525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.567817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.568100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.569494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.569962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.570984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.571288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.571740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.572152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.572750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 47/92] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:08 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:08.573229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.573508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.575036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.601373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 46/93] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:08 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:08.602474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:08.726127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:08.734188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:08.764608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32975fa6-9750-4330-adf4-dac0f9e15689 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.814014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.814707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.815002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.815286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.817268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a6135-746a-4ad8-95f2-2beaf72bde04 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:08.862710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:08.863714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:08.864194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:08.864690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:08.866546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] RBAC: Adding query filter params (role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=97ea2dd3c47c492b95e460c97de31345) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:08.867094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.868779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:08.874951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 47/94] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:08 2022] GET /identity/v3/role_assignments?user.id=97ea2dd3c47c492b95e460c97de31345&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:08.912520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:08.914396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:09.003391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:09.025967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 48/95] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:09.031873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:09.157851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4ce1-8b00-48a8-9a9d-51351295c46c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:09.177559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:09.197946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320983bd-b3f4-4ed6-ac7d-41b87d6d2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:09.198845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320983bd-b3f4-4ed6-ac7d-41b87d6d2e86 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:09.199136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320983bd-b3f4-4ed6-ac7d-41b87d6d2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:09.199416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320983bd-b3f4-4ed6-ac7d-41b87d6d2e86 None admin] PATH_INFO: `/v3/projects/18b2402a51724349ac9027ef57f32c61` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:09.200645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320983bd-b3f4-4ed6-ac7d-41b87d6d2e86 None admin] RBAC: Authorizing `identity:get_project(project_id=18b2402a51724349ac9027ef57f32c61)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:09.202112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320983bd-b3f4-4ed6-ac7d-41b87d6d2e86 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:09.203864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 48/96] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:09 2022] GET /identity/v3/projects/18b2402a51724349ac9027ef57f32c61 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:09.209890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:09.330983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fc7c-278a-4dd2-a715-34fa4e513ed5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:09.373606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:09.374275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:09.374563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:09.374872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:09.375717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=18b2402a51724349ac9027ef57f32c61, user.id=97ea2dd3c47c492b95e460c97de31345) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:09.376032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:09.377484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd0c6c-00c7-44d2-a736-0d8bc9ba8784 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:09.383504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 49/97] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:09 2022] GET /identity/v3/role_assignments?user.id=97ea2dd3c47c492b95e460c97de31345&scope.project.id=18b2402a51724349ac9027ef57f32c61&role.id=8182238074534b76a65d54c2edbeffad => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:10.405193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51524bfe-2743-4ec3-8907-44bfb20c0717 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:10.405691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51524bfe-2743-4ec3-8907-44bfb20c0717 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:10.405762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51524bfe-2743-4ec3-8907-44bfb20c0717 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:10.407454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 49/98] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:10.412494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063b404-c980-4f67-9c0a-cd49db0758c7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:10.413374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063b404-c980-4f67-9c0a-cd49db0758c7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:10.413858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063b404-c980-4f67-9c0a-cd49db0758c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:10.435607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9063b404-c980-4f67-9c0a-cd49db0758c7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:10.437140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9063b404-c980-4f67-9c0a-cd49db0758c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:10.490370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:10.523222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 50/99] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:10.557389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51524bfe-2743-4ec3-8907-44bfb20c0717 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:10.661080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461498-2c18-43e4-b036-78e0fd9e630b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:10.661392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461498-2c18-43e4-b036-78e0fd9e630b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:10.661664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461498-2c18-43e4-b036-78e0fd9e630b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:10.663083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 51/100] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:10.673274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51524bfe-2743-4ec3-8907-44bfb20c0717 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:10.676151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:10.676451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:10.676743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:10.708624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:10.709431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:10.709716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:10.709993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:10.716133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:10.717353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:10.721144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:10.729444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:10.730276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:10.731056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 50/101] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:10.737004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:10.785279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:10.844025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42744-e438-400e-8c2d-f0244cc54f6a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:10.879974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:10.881012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:10.881509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:10.881952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:10.882608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 52/102] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:10.883009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:10.883457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:10.885003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:10.894038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 51/103] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:10.900291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:10.910593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.017838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008d24d-bc22-437a-b67e-97b05bb88c3c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.028435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:11.058708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.059524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.059813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.060211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.071299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:11.079895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:11.080765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:11.081503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 52/104] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:11.089441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409fb3-17d5-450e-948d-d97008f58afe None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.127910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:11.191798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.192686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.192973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.196138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.198881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:11.200345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:11.204715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 53/105] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:10 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:11.211202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.324770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db48f4-3ed4-408c-89f6-6d5fc17bee4f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.405497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc95bb-1018-4fb7-96d6-8b3d9c896fa0 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.420754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.421439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.421730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.422010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.422807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:11.423121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:11.424502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:11.441777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 53/106] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:11.447451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.472042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.473409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.473697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.474389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.475762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:11.477236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:11.488325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 54/107] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:11 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:11.494458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.536571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:11.592680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:11.691293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adc978-0013-49fb-b460-d83e3712c844 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.717867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c672841-b97f-4202-9a24-9016e5cf8d88 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.729780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f9420-5b31-402d-b51d-8c25f1be2d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.731389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f9420-5b31-402d-b51d-8c25f1be2d19 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.731819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f9420-5b31-402d-b51d-8c25f1be2d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.732238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f9420-5b31-402d-b51d-8c25f1be2d19 None admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.733634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f9420-5b31-402d-b51d-8c25f1be2d19 None admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:11.735299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f9420-5b31-402d-b51d-8c25f1be2d19 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:11.737956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 54/108] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:11 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:11.739463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6451-568d-44cb-a493-b57214d589dc None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.739895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6451-568d-44cb-a493-b57214d589dc None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.740336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6451-568d-44cb-a493-b57214d589dc None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.741709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 55/109] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:11.746678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6451-568d-44cb-a493-b57214d589dc None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.800500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e994-9c2a-4f62-a322-cb2c3e303b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.801405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e994-9c2a-4f62-a322-cb2c3e303b1b None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.801693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e994-9c2a-4f62-a322-cb2c3e303b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.801974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e994-9c2a-4f62-a322-cb2c3e303b1b None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400/users/97ea2dd3c47c492b95e460c97de31345/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.805596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e994-9c2a-4f62-a322-cb2c3e303b1b None admin] RBAC: Authorizing `identity:create_grant(project_id=95b15952550141d9b3f8fec2905c4400, user_id=97ea2dd3c47c492b95e460c97de31345, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:11.807127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e994-9c2a-4f62-a322-cb2c3e303b1b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:11.835913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 55/110] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:11 2022] PUT /identity/v3/projects/95b15952550141d9b3f8fec2905c4400/users/97ea2dd3c47c492b95e460c97de31345/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:11.837917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.838221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.838510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.873984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:11.875379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:11.886949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:11.902393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6451-568d-44cb-a493-b57214d589dc None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:11.960828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 56/111] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:11.963337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:11.964362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:11.964799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:11.965204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:11.971593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:11.973325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:11.975692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 56/112] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:11 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:11.981447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:11.996798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.018715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:12.095287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5d26-396f-403f-8d41-cceb8de507b1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.132604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.134660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.135194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.135671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400/users/a762106a78c649808cb7ea162b5997db/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.137985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] RBAC: Authorizing `identity:create_grant(project_id=95b15952550141d9b3f8fec2905c4400, user_id=a762106a78c649808cb7ea162b5997db, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:12.143049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:12.160199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 57/113] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:11 2022] PUT /identity/v3/projects/95b15952550141d9b3f8fec2905c4400/users/a762106a78c649808cb7ea162b5997db/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:12.165483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93db5a-0b36-4fea-9303-14c5ca85d9ed None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.230149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b395a-8688-4edb-8d1f-a385474f4855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.230925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b395a-8688-4edb-8d1f-a385474f4855 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.231214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b395a-8688-4edb-8d1f-a385474f4855 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.231502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b395a-8688-4edb-8d1f-a385474f4855 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.247445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b395a-8688-4edb-8d1f-a385474f4855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-f67b395a-8688-4edb-8d1f-a385474f4855 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:12.255697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:12.256533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:12.257281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 57/114] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:12.265247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.288444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a4cb-5e74-44c8-b199-2735fe0eb586 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.289141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a4cb-5e74-44c8-b199-2735fe0eb586 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.289141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a4cb-5e74-44c8-b199-2735fe0eb586 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.291445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 58/115] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:12.296981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bf52f-63e1-4e12-aacc-b8695e986d87 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.297274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bf52f-63e1-4e12-aacc-b8695e986d87 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.297554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bf52f-63e1-4e12-aacc-b8695e986d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.347813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-229bf52f-63e1-4e12-aacc-b8695e986d87 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:12.349052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-229bf52f-63e1-4e12-aacc-b8695e986d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:12.393749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097eba6-fee4-4256-b62f-1c5673db43fa None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.403055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:12.414519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:12.438585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.439741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.440231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.440858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.441770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:12.442234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:12.444844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5d642-ccc0-44d0-be95-b7ff791dfbea None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:12.456673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 58/116] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:12.459777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2781eb-9a4e-4146-a53b-c61c1abf6924 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.460226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2781eb-9a4e-4146-a53b-c61c1abf6924 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.460651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2781eb-9a4e-4146-a53b-c61c1abf6924 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.462417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 59/117] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:12.465454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2781eb-9a4e-4146-a53b-c61c1abf6924 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.501986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 59/118] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:12.503276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.503567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.503840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.521417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:12.522636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:12.606500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2781eb-9a4e-4146-a53b-c61c1abf6924 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.610976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 60/119] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:12.611951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.650967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.652126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.652572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.652998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.654995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:12.657117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:12.659692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 60/120] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:12 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:12.660754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.690493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:12.710455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:12.728152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1056348-cdf0-4afb-969e-3b8392f8b56a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.766168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.767427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.767717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.767998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.780468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:12.787315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8dbb4-8fd8-4232-87a7-283400aafedc None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:12.788718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:12.789535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:12.791147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 61/121] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:12.791749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.842024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f46dd-1e42-4762-b18f-7ddc92b13f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.843268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f46dd-1e42-4762-b18f-7ddc92b13f6a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.843562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f46dd-1e42-4762-b18f-7ddc92b13f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.844137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f46dd-1e42-4762-b18f-7ddc92b13f6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.859738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f46dd-1e42-4762-b18f-7ddc92b13f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-fd0f46dd-1e42-4762-b18f-7ddc92b13f6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:12.870643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:12.872530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 61/122] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:12.874138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c3e09-0b16-4d84-a666-57a8cbd29d64 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.875222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c3e09-0b16-4d84-a666-57a8cbd29d64 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.875222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c3e09-0b16-4d84-a666-57a8cbd29d64 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.880250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 62/123] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:12.880250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c3e09-0b16-4d84-a666-57a8cbd29d64 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:12.909009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ee4fa-b4a2-4e0b-bfd4-e37c98a1a350 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:12.946976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f875e32-8e5e-45b9-99ee-7b6e201749ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:12.948004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f875e32-8e5e-45b9-99ee-7b6e201749ff None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.948399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f875e32-8e5e-45b9-99ee-7b6e201749ff None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.948773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f875e32-8e5e-45b9-99ee-7b6e201749ff None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.950143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f875e32-8e5e-45b9-99ee-7b6e201749ff None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:12.951837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f875e32-8e5e-45b9-99ee-7b6e201749ff None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:12.958437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 62/124] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:12 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:12.959947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:12.960381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:12.960820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:12.978228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:12.983754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:12.985201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:12.998485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.002056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c3e09-0b16-4d84-a666-57a8cbd29d64 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.041109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.042102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.042576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.043071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.044019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:13.044503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.046355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.056759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 63/125] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:13.057871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.077545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 63/126] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:13.077545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.187779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14102131-f29f-401a-83ee-8c706c88eb10 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.210187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572e7ce-3f64-4b23-b2f4-2073fbafefaa None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.228535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.229581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.229941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.230713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.231600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:13.231982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.233486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.250284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 64/127] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:13.250284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.250476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.253894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.255055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.255569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.256047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400/users/97ea2dd3c47c492b95e460c97de31345/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.258515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] RBAC: Authorizing `identity:create_grant(project_id=95b15952550141d9b3f8fec2905c4400, user_id=97ea2dd3c47c492b95e460c97de31345, role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.260761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.285251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 64/128] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:13 2022] PUT /identity/v3/projects/95b15952550141d9b3f8fec2905c4400/users/97ea2dd3c47c492b95e460c97de31345/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:13.286344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.291861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.432216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb4738-176f-4808-bcb7-54b90600931d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.485054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bf4bf-57df-4a5a-8b1e-6064aa942447 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.502127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.503148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.503442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.503727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.505797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.507507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.509936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 65/129] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:13 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:13.510513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.532759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.534223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.534765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.535339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.536721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.538388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.540641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 65/130] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:13.541676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.589077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.624733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e206f34-84ba-4a1c-98d5-bc7e44ddb231 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.629035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.660546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1cc7-9375-44a9-b47e-ac79ca15ece4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.666378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.667527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.667995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.668450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] PATH_INFO: `/v3/users/24c517298a6e430696d94fefcfc5784b` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.683774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] RBAC: Authorizing `identity:get_user(user_id=24c517298a6e430696d94fefcfc5784b)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.685606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.687688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 66/131] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:13 2022] GET /identity/v3/users/24c517298a6e430696d94fefcfc5784b => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:13.688630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.702169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.703096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.703346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.703624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] PATH_INFO: `/v3/projects/18b2402a51724349ac9027ef57f32c61` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.704916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] RBAC: Authorizing `identity:get_project(project_id=18b2402a51724349ac9027ef57f32c61)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.706421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.708597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 66/132] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:13 2022] GET /identity/v3/projects/18b2402a51724349ac9027ef57f32c61 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:13.709476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.795118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10abcd-4858-4d9c-819c-113652fae3db None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.812324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ebf9a-088c-46b8-aef9-16bc310eb137 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:13.829495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.835226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.836390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.837346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.837835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.839201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.841734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:13.848486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:13.849142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:13.849428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:13.849706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:13.861456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:13.876119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 67/133] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:51:13 2022] POST /identity/v3/projects => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:13.877316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:13.989377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:13.994929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:14.062659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387094e8-f358-4798-ad13-a984fcdbdd78 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:14.105275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:14.107080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.107572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.108049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] PATH_INFO: `/v3/projects/18b2402a51724349ac9027ef57f32c61/users/97ea2dd3c47c492b95e460c97de31345/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.110580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] RBAC: Authorizing `identity:create_grant(project_id=18b2402a51724349ac9027ef57f32c61, user_id=97ea2dd3c47c492b95e460c97de31345, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:14.113019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:14.135886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 68/134] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:13 2022] PUT /identity/v3/projects/18b2402a51724349ac9027ef57f32c61/users/97ea2dd3c47c492b95e460c97de31345/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:14.171642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 67/135] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:14.177563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:14.248547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:14.322592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf2e1a-5766-40b5-87e2-123fa0ea3310 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:14.363884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec1c92-dd44-46c1-8a02-8a9df7972a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:14.365038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec1c92-dd44-46c1-8a02-8a9df7972a2c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.365470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec1c92-dd44-46c1-8a02-8a9df7972a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.365889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec1c92-dd44-46c1-8a02-8a9df7972a2c None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.368642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec1c92-dd44-46c1-8a02-8a9df7972a2c None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:14.370255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec1c92-dd44-46c1-8a02-8a9df7972a2c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:14.372644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 69/136] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:14 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:14.378676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:14.422206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:14.488971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf130a-1442-4bc5-bf46-af217d20628c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:14.537205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:14.538535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.538987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.539412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.540810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] RBAC: Adding query filter params (role.id=d36c3084b96844ca9af694880d44ac15, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=24c517298a6e430696d94fefcfc5784b) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:14.541814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:14.543526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b983d-5c9a-411a-9cc0-c98a8385e7ec None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:14.555102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 68/137] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:14 2022] GET /identity/v3/role_assignments?user.id=24c517298a6e430696d94fefcfc5784b&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=d36c3084b96844ca9af694880d44ac15 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:14.600965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a31ef-af9b-4676-9092-a6c709f59327 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.601517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a31ef-af9b-4676-9092-a6c709f59327 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.601942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a31ef-af9b-4676-9092-a6c709f59327 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.604141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 70/138] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:14.610007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.610502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.610951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.631778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:14.633228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:14.718483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 69/139] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:14.757495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a31ef-af9b-4676-9092-a6c709f59327 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:14.896529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a31ef-af9b-4676-9092-a6c709f59327 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:14.908405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:14.935164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ea056-54ea-4d94-880e-3aa2f327e94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:14.935972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ea056-54ea-4d94-880e-3aa2f327e94d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.936260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ea056-54ea-4d94-880e-3aa2f327e94d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.936543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ea056-54ea-4d94-880e-3aa2f327e94d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.949599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ea056-54ea-4d94-880e-3aa2f327e94d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a61ea056-54ea-4d94-880e-3aa2f327e94d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:14.957449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:14.958259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:14.959028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 71/140] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:14.967721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:14.996175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa4bb1-7851-4dfb-a6c0-e46f9e2631c9 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:14.996518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa4bb1-7851-4dfb-a6c0-e46f9e2631c9 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:14.996818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa4bb1-7851-4dfb-a6c0-e46f9e2631c9 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:14.998290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 72/141] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:15.003455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.003746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.004026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:15.031091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:15.033766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:15.035020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:15.161186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 73/142] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:15.176770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad412b04-d8f7-4931-a672-581abe932ec2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:15.200111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:15.253247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:15.253938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.254227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.254509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:15.255301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:15.255610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:15.257001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:15.282637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 70/143] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:15.298398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:15.313812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:15.317313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa64ee2-c701-4a76-a194-4cc39f58b9cc None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:15.376232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:15.377034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.377323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.377598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:15.395795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:15.403952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:15.404929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:15.405719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 74/144] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:15.411131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:15.511989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:15.528624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce0990-5026-4e3b-b531-67260854366c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:15.596428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523f7f-2c23-4b64-b162-a021201f543f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:15.611381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:15.612289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.612581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.612864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:15.616171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:15.617708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:15.620041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 71/145] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:15 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:15.647660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:15.658073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:15.658784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.659071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.659349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:15.660111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:15.660416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:15.661805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:15.677098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 75/146] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:15.683011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:15.724828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:15.843103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca394a-5ca0-48b3-9918-1fbd48264332 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:15.886784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b171e7-94fa-4eeb-8ff8-e8f09cc0cdca None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:15.902548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:15.903458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.903737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.904010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] PATH_INFO: `/v3/users/a762106a78c649808cb7ea162b5997db` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:15.905219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] RBAC: Authorizing `identity:get_user(user_id=a762106a78c649808cb7ea162b5997db)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:15.906698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:15.908336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 76/147] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:15 2022] GET /identity/v3/users/a762106a78c649808cb7ea162b5997db => generated 247 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:15.913759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:15.962983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d75280-f757-4465-a468-0b6c67dd0333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:15.963647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d75280-f757-4465-a468-0b6c67dd0333 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:15.963934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d75280-f757-4465-a468-0b6c67dd0333 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:15.964216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d75280-f757-4465-a468-0b6c67dd0333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.021350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.068086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.088071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cf2e1-1fee-4f0a-9040-87cb15aff80e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:16.090431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d75280-f757-4465-a468-0b6c67dd0333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:16.091979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d75280-f757-4465-a468-0b6c67dd0333 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:16.183327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:16.184365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.184854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.185357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.197751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 72/148] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:16.199046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aafae1-67df-428e-9191-939427771c72 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.199334 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aafae1-67df-428e-9191-939427771c72 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.199611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aafae1-67df-428e-9191-939427771c72 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.201003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 73/149] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:16.210654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aafae1-67df-428e-9191-939427771c72 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:16.272786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.310371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aafae1-67df-428e-9191-939427771c72 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:16.345818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40751dfc-09f9-48c6-af90-266ee96527ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:16.346722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751dfc-09f9-48c6-af90-266ee96527ba None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.347010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751dfc-09f9-48c6-af90-266ee96527ba None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.347286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40751dfc-09f9-48c6-af90-266ee96527ba None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.347770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:16.348469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40751dfc-09f9-48c6-af90-266ee96527ba None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:16.349670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:16.349991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40751dfc-09f9-48c6-af90-266ee96527ba None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:16.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 74/150] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:16.358650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.358650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.358650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.369824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:16.371038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:16.442739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 75/151] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:16.442739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:16.547493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.548136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.566809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 77/152] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:16.567654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:16.626083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912774ce-055d-4235-85bf-4e00b8b62364 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:16.694956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:16.695536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.695815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.696087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.697560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=97ea2dd3c47c492b95e460c97de31345) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:16.697876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:16.699345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:16.715717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 76/153] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/role_assignments?user.id=97ea2dd3c47c492b95e460c97de31345&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=8182238074534b76a65d54c2edbeffad => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:16.716546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:16.762543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c8b42-90dc-4ab3-8d9c-2251cc463e53 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:16.836374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:16.837286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.837571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.837842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.865179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:16.883387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:16.884372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:16.885331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 78/154] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:16.892138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:16.923341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0736c6-8eab-483c-86fc-e03251dee903 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:16.951485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.961916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:16.963651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:16.964124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:16.964571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:16.966069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:16.967599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:16.969403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 77/155] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:16.979819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:16.979969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.065238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623184f8-6bb1-4eb9-86c9-2fe97315b0b1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.114979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.116286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.117042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.117528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.119035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:17.119610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:17.121342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:17.131872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 79/156] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:17.138015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.204926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a13211-fcee-408d-af84-24ca89278834 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.245185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.245855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.246137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.246408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.247305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=a762106a78c649808cb7ea162b5997db) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:17.247615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:17.249085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5e75-549b-45cd-a335-c53e441008b9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:17.255630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 78/157] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:16 2022] GET /identity/v3/role_assignments?user.id=a762106a78c649808cb7ea162b5997db&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:17.256624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370638d-83c8-423a-94df-758fdb2af920 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.256911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370638d-83c8-423a-94df-758fdb2af920 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.257189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370638d-83c8-423a-94df-758fdb2af920 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.258549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 79/158] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:17.264298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7eea-041a-4052-bc73-1d16bcec1562 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.264474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.264759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.265043 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.282559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:17.284607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:17.285934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:17.310724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.312052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.312678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.313305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.315076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:17.317106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:17.319284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 80/159] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:17.327781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.368835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:17.398271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 80/160] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:17.465063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.489958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de732191-2e5a-47b6-9ce2-bee6a4e230c1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.529356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c41090-0c13-4aeb-9fad-2edfaf23c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.530016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41090-0c13-4aeb-9fad-2edfaf23c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.530302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41090-0c13-4aeb-9fad-2edfaf23c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.530580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41090-0c13-4aeb-9fad-2edfaf23c7b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.585164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:17.585591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7567-a1ef-4849-8aa1-49ec3f70bd4e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.605031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c41090-0c13-4aeb-9fad-2edfaf23c7b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:17.606526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c41090-0c13-4aeb-9fad-2edfaf23c7b6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:17.624879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ebd64-18e7-4feb-952d-b6cb9c9ab89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.625672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ebd64-18e7-4feb-952d-b6cb9c9ab89f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.625954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ebd64-18e7-4feb-952d-b6cb9c9ab89f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.626229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ebd64-18e7-4feb-952d-b6cb9c9ab89f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.636710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ebd64-18e7-4feb-952d-b6cb9c9ab89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-fd5ebd64-18e7-4feb-952d-b6cb9c9ab89f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:17.644311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:17.645180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:17.646130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 81/161] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:17.647053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cef115-7712-4cb7-87e0-46b834859745 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.647053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cef115-7712-4cb7-87e0-46b834859745 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.647194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cef115-7712-4cb7-87e0-46b834859745 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.648698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 82/162] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:17.649634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76658e90-1d8e-49d2-b449-50b661043d91 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.649916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76658e90-1d8e-49d2-b449-50b661043d91 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.650191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76658e90-1d8e-49d2-b449-50b661043d91 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.651414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 83/163] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:17.653346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76658e90-1d8e-49d2-b449-50b661043d91 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.703925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:17.709501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 81/164] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:17.710790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1407510-eded-4f1a-bb04-4a23f8cd7479 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.711114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1407510-eded-4f1a-bb04-4a23f8cd7479 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.711342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1407510-eded-4f1a-bb04-4a23f8cd7479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.726836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e1407510-eded-4f1a-bb04-4a23f8cd7479 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:17.728464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e1407510-eded-4f1a-bb04-4a23f8cd7479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:17.760147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76658e90-1d8e-49d2-b449-50b661043d91 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.793428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.794092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.794208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 82/165] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:17.794372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.794669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.795425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:17.795528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.795731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:17.795821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.796011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.797104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:17.805410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 84/166] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:17.806727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.821981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:17.823255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:17.826013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:17.888848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 83/167] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:17.889658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.909884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ae43f-e0b2-484c-94b6-acf89395ee86 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:17.921040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:17.947119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:17.948024 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:17.948308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:17.948597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:17.949826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:17.951310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:17.953011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 85/168] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:17.953775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:17.991710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bdf586-9a53-410d-946a-0fea12ab7301 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.024985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.025847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.026132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.026408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] PATH_INFO: `/v3/users/97ea2dd3c47c492b95e460c97de31345` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.027635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] RBAC: Authorizing `identity:get_user(user_id=97ea2dd3c47c492b95e460c97de31345)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.029092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.030735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 84/169] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/users/97ea2dd3c47c492b95e460c97de31345 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:18.031491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.045689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.053281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359f54f-240f-48ea-af74-cbb8038bdb64 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.085635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.086427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.087259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.087559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.097696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:18.105877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:18.106944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 86/170] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:18.107904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.133505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c6dbb-3c1a-469f-81f5-c51df8149c76 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.148036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.167433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.168220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.168502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.168777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.169958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.171436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.173060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 85/171] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:18.173822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.208996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf94e3d-4292-4abc-ba01-72fbb9a928ab None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.242803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.243392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.243675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.243953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.244794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] RBAC: Adding query filter params (role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=97ea2dd3c47c492b95e460c97de31345) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:18.245109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.246576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.247831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.252864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 87/172] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/role_assignments?user.id=97ea2dd3c47c492b95e460c97de31345&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:18.253726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.286316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb8201-80ef-4569-8b2d-f30adcdecbe1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.327634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.328406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.328689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.328970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.382712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f259a-54e8-4ad5-ba42-15fd0324a52e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.384374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.409119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.410744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.424199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.425203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.425502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.426084 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.426927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:18.427403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.428794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.439448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 88/173] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:18.440172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.497796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.522541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 86/174] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:18.523691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.545401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb1d6-2b4c-4c1e-8f18-75bfb9a1bc47 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.578434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.579138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.579425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.579711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.580428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.581861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.600893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 89/175] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:18 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:18.600893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.625799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.627902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb97ed0-fc69-4159-a670-b963a30002ee None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.668565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.669466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.669753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.670033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] PATH_INFO: `/v3/users/24c517298a6e430696d94fefcfc5784b` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.671345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] RBAC: Authorizing `identity:get_user(user_id=24c517298a6e430696d94fefcfc5784b)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.672828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.674747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 87/176] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/users/24c517298a6e430696d94fefcfc5784b => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:18.680490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.720969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694a95c-166d-4496-a6b6-41de099f1897 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.734265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.760884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.762962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.762962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.762962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] PATH_INFO: `/v3/projects/18b2402a51724349ac9027ef57f32c61` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.763453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] RBAC: Authorizing `identity:get_project(project_id=18b2402a51724349ac9027ef57f32c61)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.765552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.767227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 90/177] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/projects/18b2402a51724349ac9027ef57f32c61 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:18.772502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.797271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd364b-0180-4ca2-b7dd-9d95f6c52db4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.838222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.839160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.839443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.839798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.841631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.843116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.844824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 88/178] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:18.850651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:18.876171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:18.889585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c27f8-88c1-4531-9384-283614051f52 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.931567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:18.932600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:18.933106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:18.933605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:18.934717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=18b2402a51724349ac9027ef57f32c61, user.id=97ea2dd3c47c492b95e460c97de31345) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:18.935252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:18.936908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669e8d2-887f-4350-bac8-fe0bf0b234c4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:18.946085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 91/179] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:18 2022] GET /identity/v3/role_assignments?user.id=97ea2dd3c47c492b95e460c97de31345&scope.project.id=18b2402a51724349ac9027ef57f32c61&role.id=8182238074534b76a65d54c2edbeffad => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:18.972599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ce4eb-b60d-41bc-bcc2-f24253f64e31 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:18.992077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:19.019896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ff544-751d-45c0-88f7-dbda30c4674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:19.020738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ff544-751d-45c0-88f7-dbda30c4674c None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:19.021030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ff544-751d-45c0-88f7-dbda30c4674c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:19.021336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ff544-751d-45c0-88f7-dbda30c4674c None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400/users/24c517298a6e430696d94fefcfc5784b/roles/d36c3084b96844ca9af694880d44ac15` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:19.023569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ff544-751d-45c0-88f7-dbda30c4674c None admin] RBAC: Authorizing `identity:create_grant(project_id=95b15952550141d9b3f8fec2905c4400, user_id=24c517298a6e430696d94fefcfc5784b, role_id=d36c3084b96844ca9af694880d44ac15)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:19.025615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ff544-751d-45c0-88f7-dbda30c4674c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:19.044532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 89/180] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:18 2022] PUT /identity/v3/projects/95b15952550141d9b3f8fec2905c4400/users/24c517298a6e430696d94fefcfc5784b/roles/d36c3084b96844ca9af694880d44ac15 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:20.810090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5519fd-081b-4802-86be-097f7fe466ff None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:20.810649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5519fd-081b-4802-86be-097f7fe466ff None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:20.810729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5519fd-081b-4802-86be-097f7fe466ff None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:20.812073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 92/181] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:20.821938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0599d2c-df25-4fd0-b959-26337673e01e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:20.822386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0599d2c-df25-4fd0-b959-26337673e01e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:20.822560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0599d2c-df25-4fd0-b959-26337673e01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:20.839773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b0599d2c-df25-4fd0-b959-26337673e01e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:20.840952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b0599d2c-df25-4fd0-b959-26337673e01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:20.910124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 90/182] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:20.933802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5519fd-081b-4802-86be-097f7fe466ff None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.032276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd405-0972-4cee-a235-55b6c2bac86c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.032589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd405-0972-4cee-a235-55b6c2bac86c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.032837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd405-0972-4cee-a235-55b6c2bac86c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.034389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 91/183] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:21.039538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703628f0-294d-43ca-9cbe-dfe3dfab73c5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.040169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703628f0-294d-43ca-9cbe-dfe3dfab73c5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.040618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703628f0-294d-43ca-9cbe-dfe3dfab73c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.057609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5519fd-081b-4802-86be-097f7fe466ff None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.061946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-703628f0-294d-43ca-9cbe-dfe3dfab73c5 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:21.063383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-703628f0-294d-43ca-9cbe-dfe3dfab73c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:21.095208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8615b3a-6f03-4dd4-8f24-99aad334d77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.096110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8615b3a-6f03-4dd4-8f24-99aad334d77c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.096403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8615b3a-6f03-4dd4-8f24-99aad334d77c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.096677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8615b3a-6f03-4dd4-8f24-99aad334d77c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.107118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8615b3a-6f03-4dd4-8f24-99aad334d77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b8615b3a-6f03-4dd4-8f24-99aad334d77c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:21.114740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:21.115624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:21.116385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 93/184] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:21.117137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bab7-56bb-49f9-ac38-544fdc452c8b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.117388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bab7-56bb-49f9-ac38-544fdc452c8b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.117661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bab7-56bb-49f9-ac38-544fdc452c8b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.119614 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 94/185] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:21.120599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3a312-1ce0-4093-aa22-54b4307aabf9 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.120784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3a312-1ce0-4093-aa22-54b4307aabf9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.121059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3a312-1ce0-4093-aa22-54b4307aabf9 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.122255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 95/186] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:21.126681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3a312-1ce0-4093-aa22-54b4307aabf9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.138899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 92/187] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:21.140172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8997d9-bcf3-478a-ac48-9d43eca9f54c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.140462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8997d9-bcf3-478a-ac48-9d43eca9f54c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.140739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8997d9-bcf3-478a-ac48-9d43eca9f54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.143928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.157118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bb8997d9-bcf3-478a-ac48-9d43eca9f54c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:21.158304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bb8997d9-bcf3-478a-ac48-9d43eca9f54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:21.185839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.227316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 93/188] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:21.228560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.228838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.229114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.233181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3a312-1ce0-4093-aa22-54b4307aabf9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.251630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:21.252810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:21.269655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.270320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.270608 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.270911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.273365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:21.273365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.273365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:21.281731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 96/189] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:21.282343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.326275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 94/190] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:21.327517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.358745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.383389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc798f1f-933a-48c7-8ff3-7b58421b1843 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.414670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.416976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.417786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.418070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.418347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.425228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442729a-85c1-48c1-bc3d-2c5c3106e80b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.428307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:21.436910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:21.437771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:21.438525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 97/191] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:21.440295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.458990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.459777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.460061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.460337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.461485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.462970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:21.464705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 95/192] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:21.465322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.542134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bb21c-9a27-404f-a7a3-dd5f91098efa None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.562746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fecbef7-e878-4307-9cd0-554c4fcc40e5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.574867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.575741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.576026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.576354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] PATH_INFO: `/v3/users/916361916a1743a0a406201a4f0da748` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.578238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.592730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] RBAC: Authorizing `identity:get_user(user_id=916361916a1743a0a406201a4f0da748)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.594181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:21.595893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 98/193] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/users/916361916a1743a0a406201a4f0da748 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:21.598777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.598906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.598999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.598999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.598999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.609436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:21.617043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:21.617892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:21.619280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 96/194] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:21.619912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.641076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.702775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10375335-431b-46fc-b622-cdb00c965caa None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.717557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48755332-a126-48a2-9b1e-b6642657c779 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.735227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.735892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.736173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.736444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.737190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:21.737488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.738892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:21.747480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 99/195] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:21.748328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.751029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.751822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.752117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.752401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.753117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.754569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:21.780662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 97/196] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:21 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:21.781462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.807434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.868383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58780171-8ed8-4ec0-8480-90143d4e3692 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.874836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:21.901294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c210d6-abd3-4dff-9fc1-b66fb6d3eee9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:21.909242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.909907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.910214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.910470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.938732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:21.939409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:21.939691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:21.939975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:21.940738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:21.941056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.942440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:21.951105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 98/197] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:21.951934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:21.989185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:21.990772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.061894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.076808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f664e-1ebb-48e3-8f34-60005c24b3c0 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.113039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.114019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.114300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.114578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] PATH_INFO: `/v3/users/916361916a1743a0a406201a4f0da748` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.115815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] RBAC: Authorizing `identity:get_user(user_id=916361916a1743a0a406201a4f0da748)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.116879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 100/198] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:22.117636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.117913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.123623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 99/199] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:21 2022] GET /identity/v3/users/916361916a1743a0a406201a4f0da748 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:22.124745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.134785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.240400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6478e-1ac2-4b97-94e6-37510907fbe5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.249674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a6859-9381-471a-803c-d49ab5eeb721 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.290301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.291908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.292750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.293208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] PATH_INFO: `/v3/users/24c517298a6e430696d94fefcfc5784b` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.295189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] RBAC: Authorizing `identity:get_user(user_id=24c517298a6e430696d94fefcfc5784b)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.297514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.299369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 101/200] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/users/24c517298a6e430696d94fefcfc5784b => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:22.304497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.304723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.304723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.304723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.304723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.305044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.306215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.308387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 100/201] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:22.313840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.336253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.410834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.427994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ae7a8-8ea1-45cb-b2e0-d3930a3d39e0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.449713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5fd5f-6fc9-4281-8ee3-b9b377f67299 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.465059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.465842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.466245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.466647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.485800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.486760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.487216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.487659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.536230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.537849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.561961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.563689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.565381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.642999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 102/202] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:22.643987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.650936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.679546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 101/203] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:22.681397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.746229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94998490-6dc5-422b-a1fc-7a87141c6867 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.769132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:22.781102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.781979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.782107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.782336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.783407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=916361916a1743a0a406201a4f0da748) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:22.783699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.785756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.789623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db887b11-39ba-4914-9908-e85de9558027 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.792170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 103/204] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/role_assignments?user.id=916361916a1743a0a406201a4f0da748&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:22.792738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.828981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:22.830323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:22.830843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:22.831361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:22.832770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:22.834521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:22.836448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 102/205] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:22.845752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:22.945206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4cf56-df99-4ee2-9310-28d139670f80 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:22.964952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:23.021606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.022576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.022910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.023194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.024475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.026068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.028250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 104/206] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:23.033882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:23.044886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83076e-ed0b-442b-845e-d19eb5623282 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.100132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:23.124999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.125666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.126002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.126354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.127241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] RBAC: Adding query filter params (role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=916361916a1743a0a406201a4f0da748) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:23.127556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.129011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011dacb-d84f-427f-aced-a45110ac9759 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.142349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 103/207] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:22 2022] GET /identity/v3/role_assignments?user.id=916361916a1743a0a406201a4f0da748&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:23.187374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e8d92-726d-478f-98b2-877e0b63428c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:23.245179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:23.245771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:23.246064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:23.246341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:23.247230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] RBAC: Adding query filter params (role.id=d36c3084b96844ca9af694880d44ac15, scope.project.id=95b15952550141d9b3f8fec2905c4400, user.id=24c517298a6e430696d94fefcfc5784b) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:23.247545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:23.249008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08e664-8a51-4c1b-a4d9-9d5e26ba5e72 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:23.260002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 105/208] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:23 2022] GET /identity/v3/role_assignments?user.id=24c517298a6e430696d94fefcfc5784b&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=d36c3084b96844ca9af694880d44ac15 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:24.042970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23170b9a-ea89-4984-8f8d-0ce6c8f4ed2e None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.043985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23170b9a-ea89-4984-8f8d-0ce6c8f4ed2e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.044498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23170b9a-ea89-4984-8f8d-0ce6c8f4ed2e None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.046419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 104/209] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:24.052537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba16827-1acd-40c4-929f-89ea56391b16 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.053579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba16827-1acd-40c4-929f-89ea56391b16 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.054080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba16827-1acd-40c4-929f-89ea56391b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.072633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4ba16827-1acd-40c4-929f-89ea56391b16 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:24.074156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4ba16827-1acd-40c4-929f-89ea56391b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:24.143951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 106/210] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:24.166870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23170b9a-ea89-4984-8f8d-0ce6c8f4ed2e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.253334 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f00a-3c5b-4c8a-a57f-0eb32b432eaf None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.253676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f00a-3c5b-4c8a-a57f-0eb32b432eaf None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.253916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f00a-3c5b-4c8a-a57f-0eb32b432eaf None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.255607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 107/211] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:24.260924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.261219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.261503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.273402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23170b9a-ea89-4984-8f8d-0ce6c8f4ed2e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.285908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:24.287392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:24.306405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.315204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.316262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.316590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.316930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.331101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:24.337197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:24.341575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:24.342604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:24.344012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 105/212] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:24.348875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.369249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 108/213] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:24.395260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.454701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d9e72-be80-4527-a180-7442b4cf7511 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.488904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.489602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.489860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.490150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.490930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:24.491240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.492692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.502678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a94f33-4801-47e1-b263-9b7fd514975f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.506806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 106/214] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:24.507697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.539023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.541331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.542399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.543001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.543481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.545618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.547431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.554171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 109/215] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:24.555336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.610960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.626330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa12f5-8444-4b08-97b8-ca31a338e632 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.661145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.662059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.662342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.662647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] PATH_INFO: `/v3/users/d793c666354a42aa8fb101239a4d368a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.679372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9cc22-bf6e-4a22-b783-34b01d688075 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.679884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] RBAC: Authorizing `identity:get_user(user_id=d793c666354a42aa8fb101239a4d368a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.681954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.689577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 107/216] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/users/d793c666354a42aa8fb101239a4d368a => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:24.690417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:24.722185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be3f97-86cb-4ef9-8460-b841675480aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.723370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3f97-86cb-4ef9-8460-b841675480aa None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.724154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3f97-86cb-4ef9-8460-b841675480aa None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.724928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be3f97-86cb-4ef9-8460-b841675480aa None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.726008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be3f97-86cb-4ef9-8460-b841675480aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.728782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be3f97-86cb-4ef9-8460-b841675480aa None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.763192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 110/217] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:24.797462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:24.814970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599709c3-d845-4e00-9dbe-31b0c61a8241 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:24.854982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:24.855652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.855938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.856217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.936586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:24.938015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:24.989894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48746387-6e40-406e-9e5c-365c9ca9488e None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.990259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48746387-6e40-406e-9e5c-365c9ca9488e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.990503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48746387-6e40-406e-9e5c-365c9ca9488e None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:24.992151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 111/218] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:24.997418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:24.997706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:24.997990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.019807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:25.021016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:25.045556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:25.049998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 108/219] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:25.055776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.098708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 112/220] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:25.117992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.155093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:25.172256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b58c7-065e-4072-9ebd-dbfa2c93248f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.218604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.219444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.219729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.220006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.221777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:25.223780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:25.225853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 109/221] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:25.231263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.256139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf88ac7-676e-4e39-a3be-ced80ac4d483 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.325706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:25.355342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb2396-2d4b-4586-8dd5-fe249b0482ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.356164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2396-2d4b-4586-8dd5-fe249b0482ea None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.356448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2396-2d4b-4586-8dd5-fe249b0482ea None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.356722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb2396-2d4b-4586-8dd5-fe249b0482ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.376089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c098ee-6727-4108-9b97-3786bb34a2f3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.376668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb2396-2d4b-4586-8dd5-fe249b0482ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-3dbb2396-2d4b-4586-8dd5-fe249b0482ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:25.387104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:25.387993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:25.388751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:25.390787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 113/222] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:25.392123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2edaa4-c9f3-4fef-a950-152e45590e14 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.392407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2edaa4-c9f3-4fef-a950-152e45590e14 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.392681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2edaa4-c9f3-4fef-a950-152e45590e14 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.394047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 114/223] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:25.395961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2edaa4-c9f3-4fef-a950-152e45590e14 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.424142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.425041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.425326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.425601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.426858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=d793c666354a42aa8fb101239a4d368a) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:25.427171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:25.428628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3b8c6-830a-4796-b055-c2f9daea3851 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:25.436821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 110/224] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/role_assignments?user.id=d793c666354a42aa8fb101239a4d368a&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=8182238074534b76a65d54c2edbeffad => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:25.438444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.439104 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.439852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.462796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:25.464506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:25.547557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:25.572369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2edaa4-c9f3-4fef-a950-152e45590e14 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.616102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 111/225] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:25.635914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.636617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.636903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.637184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.637961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:25.638269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:25.639666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.639774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:25.658665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 115/226] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:25.660942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.741232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:25.807015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117814c-c60c-4ccf-afb1-d343df426839 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.846139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.847417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.847930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.848408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] PATH_INFO: `/v3/users/916361916a1743a0a406201a4f0da748` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.849895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] RBAC: Authorizing `identity:get_user(user_id=916361916a1743a0a406201a4f0da748)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:25.852053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:25.854298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 116/227] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/users/916361916a1743a0a406201a4f0da748 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:25.858561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ed3ec-db11-4810-afa7-a053c75e53fb None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:25.859828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:25.891114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:25.939567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:25.940839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:25.941378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:25.941880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:25.955366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:25.964024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:25.964884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:25.965644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:25.968175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 112/228] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:25.974010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:26.033398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d52ba-d639-4c76-83f3-80747021fa6d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:26.091775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97609ae4-06aa-4902-8d8b-ed96fd2830bb None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:26.102060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:26.131552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:26.132584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:26.133065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:26.133564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:26.134530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:26.135173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:26.136903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:26.145875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 113/229] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:26.147322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:26.148216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:26.148506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:26.148787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:26.150075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:26.151513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:26.151842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:26.153553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 117/230] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:25 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:26.159246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:26.262222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1512e-b238-4427-996e-4b783c6757b8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:26.350174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:26.351424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:26.351953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:26.353109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] PATH_INFO: `/v3/users/916361916a1743a0a406201a4f0da748` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:26.355626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] RBAC: Authorizing `identity:get_user(user_id=916361916a1743a0a406201a4f0da748)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:26.357281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:26.362652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 114/231] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:26 2022] GET /identity/v3/users/916361916a1743a0a406201a4f0da748 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:26.369003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:26.402990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:26.421124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007e37-d87e-4f75-bfe5-cbe7ade6bd19 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:26.438939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:26.487924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:26.488830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:26.489116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:26.491878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/916361916a1743a0a406201a4f0da748/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:26.503402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] RBAC: Authorizing `identity:create_grant(project_id=9a2df333a2bd4f919ac91e69ef586e3c, user_id=916361916a1743a0a406201a4f0da748, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:26.505096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:26.527242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e1021-2571-48d2-a125-a0a63c61b502 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:26.536744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 118/232] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:26 2022] PUT /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/916361916a1743a0a406201a4f0da748/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:26.565714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06d8b1-f9d3-40dc-8ef5-f349943acdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:26.566577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06d8b1-f9d3-40dc-8ef5-f349943acdaa None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:26.566890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06d8b1-f9d3-40dc-8ef5-f349943acdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:26.567162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06d8b1-f9d3-40dc-8ef5-f349943acdaa None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:26.568415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06d8b1-f9d3-40dc-8ef5-f349943acdaa None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:26.570109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06d8b1-f9d3-40dc-8ef5-f349943acdaa None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:26.572546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 115/233] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:26 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:26.578444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:26.744216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4e18b-9708-4d8c-a61a-269f0e0fa1da None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:26.792241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f7083-cf1b-4cae-a4d6-fe8c5a922c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:26.793144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f7083-cf1b-4cae-a4d6-fe8c5a922c06 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:26.793423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f7083-cf1b-4cae-a4d6-fe8c5a922c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:26.793696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f7083-cf1b-4cae-a4d6-fe8c5a922c06 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/916361916a1743a0a406201a4f0da748/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:26.795967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f7083-cf1b-4cae-a4d6-fe8c5a922c06 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a2df333a2bd4f919ac91e69ef586e3c, user_id=916361916a1743a0a406201a4f0da748, role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:26.797408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f7083-cf1b-4cae-a4d6-fe8c5a922c06 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:26.814929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 119/234] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:26 2022] PUT /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/916361916a1743a0a406201a4f0da748/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:27.269622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dae4aa-d8b9-4ffd-bbee-25fdd324f7a2 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.270127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dae4aa-d8b9-4ffd-bbee-25fdd324f7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.270227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dae4aa-d8b9-4ffd-bbee-25fdd324f7a2 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.276706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 116/235] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:27 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:27.282242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d7cb2-3198-4149-b226-87202a09d80b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.283123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d7cb2-3198-4149-b226-87202a09d80b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.283621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d7cb2-3198-4149-b226-87202a09d80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.306900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-185d7cb2-3198-4149-b226-87202a09d80b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:27.308419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-185d7cb2-3198-4149-b226-87202a09d80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:27.315624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:27.396431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 120/236] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:27.431954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dae4aa-d8b9-4ffd-bbee-25fdd324f7a2 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:27.496958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d553-0729-411d-8db1-f28a42c16946 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.497609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d553-0729-411d-8db1-f28a42c16946 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.498085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1d553-0729-411d-8db1-f28a42c16946 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.499693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 121/237] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:27.504736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.505277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.505752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.531757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:27.532860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:27.567117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dae4aa-d8b9-4ffd-bbee-25fdd324f7a2 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:27.575715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:27.604872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:27.605879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.606234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.606785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.617468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:27.625219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:27.626068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:27.626874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 117/238] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:27.632238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:27.680675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 122/239] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:27.720052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:27.747881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c3aaa-eee6-483b-aeb3-4434ccfaa77b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:27.784776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:27.785396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.785680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.785959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.789997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:27.789997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:27.789997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:27.802637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 118/240] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:27.802913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:27.854959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:27.856283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:27.932563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8d14-1ebd-4975-87fe-22dbd3d8c408 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:27.935694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e5733-3dab-4910-9b4a-3a03eafc62f4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:27.974908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759f1f-e0a5-4ded-82d3-ec2e16a2a782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:27.975409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cf751-38e2-4ff3-b597-f89df41bd83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:27.975715 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759f1f-e0a5-4ded-82d3-ec2e16a2a782 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.976002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759f1f-e0a5-4ded-82d3-ec2e16a2a782 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.976278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf751-38e2-4ff3-b597-f89df41bd83f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.976399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759f1f-e0a5-4ded-82d3-ec2e16a2a782 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.976549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf751-38e2-4ff3-b597-f89df41bd83f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.976926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf751-38e2-4ff3-b597-f89df41bd83f None admin] PATH_INFO: `/v3/users/38ed7222c0c34029988f44df93b8262c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.977523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759f1f-e0a5-4ded-82d3-ec2e16a2a782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:27.979195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759f1f-e0a5-4ded-82d3-ec2e16a2a782 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:27.981943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 123/241] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:27.983181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028c8d0-be6f-484c-88e5-d696cc588623 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:27.983467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028c8d0-be6f-484c-88e5-d696cc588623 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:27.983747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028c8d0-be6f-484c-88e5-d696cc588623 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:27.984978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 124/242] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:27.990933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028c8d0-be6f-484c-88e5-d696cc588623 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:27.999502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cf751-38e2-4ff3-b597-f89df41bd83f None admin] RBAC: Authorizing `identity:get_user(user_id=38ed7222c0c34029988f44df93b8262c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.001023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cf751-38e2-4ff3-b597-f89df41bd83f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.003553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 119/243] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:27 2022] GET /identity/v3/users/38ed7222c0c34029988f44df93b8262c => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:28.005260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.005543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.005821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.026353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:28.028017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:28.111626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 120/244] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:28.112636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.139398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028c8d0-be6f-484c-88e5-d696cc588623 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.149820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:28.181579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.182522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.182968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.183401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.184375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.186060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.195661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:28.218881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 125/245] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:27 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:28.220177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.241740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e77ed-44a2-45f9-83f2-de14c25b00fe None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.292824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.293497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.293786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.294064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.423616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.426126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.432144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4adbb6-17dd-4133-b93d-964872c14a6a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.451206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:28.483871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.484680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.484966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.485242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.500856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:28.512572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:28.513673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:28.514972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 126/246] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:28.520514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.566368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 121/247] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:28.579253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.594213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:28.654819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c86b-6846-4b36-aed2-8274340863f3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.700871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.701610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.702513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.702642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.703801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:28.704187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.705846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.715336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb8cb-b6c1-46ef-b744-0ebdbb897660 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.717873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 127/248] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:28.723365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.735689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:28.760900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.762768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.763226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.763915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.765946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.767869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.771237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 122/249] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:28.781268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.860924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb39aaa-c4b6-44e2-8d63-c1602f6a6b25 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.882824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:28.904401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c80f5-6737-41be-9869-f30ba801bd9b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:28.907854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.909776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.910264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.910784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] PATH_INFO: `/v3/users/d793c666354a42aa8fb101239a4d368a` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.912209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] RBAC: Authorizing `identity:get_user(user_id=d793c666354a42aa8fb101239a4d368a)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.914332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.917127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 128/250] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/users/d793c666354a42aa8fb101239a4d368a => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:28.924553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:28.956381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:28.957301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:28.957491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:28.957887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:28.958882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] RBAC: Adding query filter params (role.id=d36c3084b96844ca9af694880d44ac15, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=38ed7222c0c34029988f44df93b8262c) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:28.959284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:28.960958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca6158-1c10-44af-9159-1b126dc955a2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:28.967627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 123/251] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/role_assignments?user.id=38ed7222c0c34029988f44df93b8262c&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=d36c3084b96844ca9af694880d44ac15 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:29.032972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8838762-0e6a-4f1f-8d8c-7ac701e9650c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.033386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8838762-0e6a-4f1f-8d8c-7ac701e9650c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.033755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8838762-0e6a-4f1f-8d8c-7ac701e9650c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.035657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 124/252] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:29.044046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.044422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.044778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.061786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65833fd-8823-4bb7-b9c2-826e22ea9722 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:29.068904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:29.070508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:29.104784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:29.106718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.107211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.107687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.109655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:29.111842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:29.114194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 129/253] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:28 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:29.126956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.137714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:29.171699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 125/254] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:29.196455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.219042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:29.321196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80a76-d1ab-4d45-adb3-e3684d9bdf6f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:29.381965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552ee6b-6225-4de8-8c8d-6e5e4b44427c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:29.395351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187a68f-c0e0-4b6d-b6e4-cd561367057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:29.396663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a68f-c0e0-4b6d-b6e4-cd561367057d None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.397167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a68f-c0e0-4b6d-b6e4-cd561367057d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.397656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a68f-c0e0-4b6d-b6e4-cd561367057d None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/d793c666354a42aa8fb101239a4d368a/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.400599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187a68f-c0e0-4b6d-b6e4-cd561367057d None admin] RBAC: Authorizing `identity:create_grant(project_id=9a2df333a2bd4f919ac91e69ef586e3c, user_id=d793c666354a42aa8fb101239a4d368a, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:29.404268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187a68f-c0e0-4b6d-b6e4-cd561367057d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:29.425300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 130/255] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:29 2022] PUT /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/d793c666354a42aa8fb101239a4d368a/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:29.429611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3de3f1-7342-473b-93c4-f8db06fbe7b1 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.429611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3de3f1-7342-473b-93c4-f8db06fbe7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.429611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3de3f1-7342-473b-93c4-f8db06fbe7b1 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.429611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 131/256] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:29.434148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.434713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.434713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.455654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:29.455654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:29.466987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:29.468157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.468679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.469210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.493032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:29.507160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:29.508107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:29.508947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:29.511473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 126/257] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:29.517582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.591025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:29.602973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 132/258] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:29.643678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.682713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc417-5021-48b5-a037-72d030669beb None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:29.728615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:29.779416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:29.780118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.780407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.780684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.781464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:29.781771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:29.783204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:29.808914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 127/259] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:29.815667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.816633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfff31a-6dc6-4602-b777-7d8135fa8794 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:29.888902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:29.889791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:29.890078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:29.890393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:29.910450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:29.922493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:29.923416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:29.924302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 133/260] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:29.932403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:29.995912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:30.002779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f012d-33e7-40e4-a61d-a1b44d89425b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:30.074868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4b517-4b74-4664-ab7c-675286de1ad0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:30.076413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:30.091314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:30.092514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:30.093001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:30.093511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] PATH_INFO: `/v3/users/916361916a1743a0a406201a4f0da748` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:30.095185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] RBAC: Authorizing `identity:get_user(user_id=916361916a1743a0a406201a4f0da748)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:30.096884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:30.103970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 128/261] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:29 2022] GET /identity/v3/users/916361916a1743a0a406201a4f0da748 => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:30.107541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:30.129224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:30.129909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:30.130251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:30.130542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:30.131373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:30.131693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:30.133220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:30.146511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 134/262] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:30.152516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:30.278589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe48e44-c6c1-4057-8834-8690bbfa3b08 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:30.314371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:30.315297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:30.315585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:30.315862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] PATH_INFO: `/v3/users/916361916a1743a0a406201a4f0da748` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:30.317324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] RBAC: Authorizing `identity:get_user(user_id=916361916a1743a0a406201a4f0da748)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:30.320189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:30.321061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 135/263] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:30 2022] GET /identity/v3/users/916361916a1743a0a406201a4f0da748 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:30.327514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:30.345115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe33aa-a302-4822-adb0-b08072054535 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:30.387908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:30.411061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:30.412307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:30.412739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:30.413162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:30.490643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:30.533342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb1374-293c-4cd4-bfcb-2b368041106f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:30.559596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:30.561556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:30.663323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:30.664234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:30.665010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:30.665464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:30.736418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 129/264] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:30.744466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:30.806255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:30.808128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:30.849298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:30.856308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:30.895465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8fa9-184d-4646-837a-4675f0bb523f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:30.934674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:30.936887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:30.937182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:30.937464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:30.938728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:30.940179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:30.943370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 130/265] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:30 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:30.952386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:30.989523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 136/266] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:30.999695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:31.092924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6554-7681-4b50-99dd-affccc9d9877 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:31.156302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:31.156985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.157268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.157541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.158395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=916361916a1743a0a406201a4f0da748) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:31.158735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:31.160204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:31.166918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:31.173056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 131/267] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:30 2022] GET /identity/v3/role_assignments?user.id=916361916a1743a0a406201a4f0da748&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:31.238934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074091d-ad10-4e28-b9a9-a691e2873d6f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:31.307886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:31.314043 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f937d-c3cf-426c-9a0a-dca84f0fb7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:31.315396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f937d-c3cf-426c-9a0a-dca84f0fb7eb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.315895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f937d-c3cf-426c-9a0a-dca84f0fb7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.316376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f937d-c3cf-426c-9a0a-dca84f0fb7eb None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.317930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f937d-c3cf-426c-9a0a-dca84f0fb7eb None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:31.319697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f937d-c3cf-426c-9a0a-dca84f0fb7eb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:31.322083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 137/268] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:30 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:31.330100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:31.455685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2ff8c-dd35-46e3-b804-9ed0413b5707 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:31.504012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:31.504667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.504952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.505230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.506069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] RBAC: Adding query filter params (role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=916361916a1743a0a406201a4f0da748) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:31.506376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:31.507867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da6269-f577-4e2a-bade-cbfdc0c65a20 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:31.519482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 132/269] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:31 2022] GET /identity/v3/role_assignments?user.id=916361916a1743a0a406201a4f0da748&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:31.663590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842be2a-0f99-4d6f-b288-c09a29a2210b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.665376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842be2a-0f99-4d6f-b288-c09a29a2210b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.665862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842be2a-0f99-4d6f-b288-c09a29a2210b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.667785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 138/270] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:31.675472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fcfb2-e912-4150-9585-fdd30e673e9b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.675757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fcfb2-e912-4150-9585-fdd30e673e9b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.676199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fcfb2-e912-4150-9585-fdd30e673e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.696763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f07fcfb2-e912-4150-9585-fdd30e673e9b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:31.698207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f07fcfb2-e912-4150-9585-fdd30e673e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:31.778014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:31.786730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 133/271] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:31.816614 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842be2a-0f99-4d6f-b288-c09a29a2210b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:31.932174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842be2a-0f99-4d6f-b288-c09a29a2210b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:31.937978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654b5eb-0e86-4db2-8d72-7901bd1dd0d4 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.938262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654b5eb-0e86-4db2-8d72-7901bd1dd0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.938541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654b5eb-0e86-4db2-8d72-7901bd1dd0d4 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.943773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 134/272] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:31.946759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cf42b-b886-406b-b55a-60876ac2201b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.947331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cf42b-b886-406b-b55a-60876ac2201b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.947900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cf42b-b886-406b-b55a-60876ac2201b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:31.965836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-dc2cf42b-b886-406b-b55a-60876ac2201b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:31.967381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-dc2cf42b-b886-406b-b55a-60876ac2201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:31.983792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f9c2a-1290-4e13-83f4-ddcf851435fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:31.984925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f9c2a-1290-4e13-83f4-ddcf851435fc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:31.984925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f9c2a-1290-4e13-83f4-ddcf851435fc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:31.985181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f9c2a-1290-4e13-83f4-ddcf851435fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.000456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f9c2a-1290-4e13-83f4-ddcf851435fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0d4f9c2a-1290-4e13-83f4-ddcf851435fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:32.009072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:32.009990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:32.010792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 139/273] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:32.014687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef47f0-e1ba-4459-8fd6-6f3c2b68581a None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.014687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef47f0-e1ba-4459-8fd6-6f3c2b68581a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.014687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef47f0-e1ba-4459-8fd6-6f3c2b68581a None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.014687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 140/274] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:32.018314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef47f0-e1ba-4459-8fd6-6f3c2b68581a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.066716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 135/275] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:32.066716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.066716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.066716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.082972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:32.084394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.085509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:32.129518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef47f0-e1ba-4459-8fd6-6f3c2b68581a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.154821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 136/276] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:32.160216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.167416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.168141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.168426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.168797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.169593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:32.169950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.171525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.184781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 141/277] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:32.184781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.227472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.273900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e323f2-ee56-4be7-8c23-24d45438487b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.301763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615897c-a91c-41f1-af14-8e735138e5a2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.310608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.311433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.311716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.311992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.313310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.314730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.316369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 137/278] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:32.317173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.332685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.344213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64980d-ba3c-4c69-8276-db4d0b54fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.345011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64980d-ba3c-4c69-8276-db4d0b54fe3d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.345298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64980d-ba3c-4c69-8276-db4d0b54fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.345570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64980d-ba3c-4c69-8276-db4d0b54fe3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.355509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64980d-ba3c-4c69-8276-db4d0b54fe3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-1b64980d-ba3c-4c69-8276-db4d0b54fe3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:32.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:32.364269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:32.365225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 142/279] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:32.366006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3d4f6-daac-402a-82b9-c26c82f69422 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.366006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3d4f6-daac-402a-82b9-c26c82f69422 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.366152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3d4f6-daac-402a-82b9-c26c82f69422 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.368013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 143/280] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:32.368606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3d4f6-daac-402a-82b9-c26c82f69422 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.424999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf81f81-19b8-46e5-a1a8-49b7e2b8260f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.447725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.459379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.460250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.460537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.460820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] PATH_INFO: `/v3/users/3cbfab52e5ba4b19861aab6a9eb5be80` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.467603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3d4f6-daac-402a-82b9-c26c82f69422 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.473721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbfab52e5ba4b19861aab6a9eb5be80)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.475219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.481454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 138/281] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/users/3cbfab52e5ba4b19861aab6a9eb5be80 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:32.481454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.504621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47b060-38f4-49d5-82ed-f66aeb9ffa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.505284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47b060-38f4-49d5-82ed-f66aeb9ffa59 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.505573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47b060-38f4-49d5-82ed-f66aeb9ffa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.505838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47b060-38f4-49d5-82ed-f66aeb9ffa59 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.506540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47b060-38f4-49d5-82ed-f66aeb9ffa59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.507999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47b060-38f4-49d5-82ed-f66aeb9ffa59 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.529314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 144/282] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:32 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:32.529314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.529314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.529314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.548878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:32.550136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:32.584818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.596863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1592b-d8cd-4297-8775-b3f7d58bbba8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.628693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 145/283] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:32.629558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.649040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.650177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.650500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.650811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.651961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:32.652284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.654307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.663464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 139/284] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:32.664867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.715394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.746383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1999e50-ca3b-4a71-a165-c52fa5b1acac None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.785599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.786294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.786464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc820e96-e466-435c-8863-58e617b9a0a5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.786559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.786915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.787920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, user.id=3cbfab52e5ba4b19861aab6a9eb5be80, scope.system=all) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:32.788128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.790150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.796976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 146/285] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/role_assignments?user.id=3cbfab52e5ba4b19861aab6a9eb5be80&scope.system=all&role.id=8182238074534b76a65d54c2edbeffad => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:32.797701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.835177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.836200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.836495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.836773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.838977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:32.854506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:32.862784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:32.863726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:32.864488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 140/286] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:32.872973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.923908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30632196-5033-4d7f-a956-e6758adeacdf None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.963728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07122506-145a-4477-9731-93ed096816a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:32.964918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07122506-145a-4477-9731-93ed096816a3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:32.965348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07122506-145a-4477-9731-93ed096816a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:32.965796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07122506-145a-4477-9731-93ed096816a3 None admin] PATH_INFO: `/v3/users/38ed7222c0c34029988f44df93b8262c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:32.971635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07122506-145a-4477-9731-93ed096816a3 None admin] RBAC: Authorizing `identity:get_user(user_id=38ed7222c0c34029988f44df93b8262c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:32.973348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07122506-145a-4477-9731-93ed096816a3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:32.975539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 147/287] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/users/38ed7222c0c34029988f44df93b8262c => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:32.981815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07122506-145a-4477-9731-93ed096816a3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:32.984199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6faaf-9276-47bd-9897-35b5903139b3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:32.986976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:33.033993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:33.034697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:33.034987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:33.035266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:33.036603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:33.036915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:33.038813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:33.050725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 141/288] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:33.056171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:33.105272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:33.108823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07122506-145a-4477-9731-93ed096816a3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:33.145316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:33.146422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:33.146883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:33.147407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:33.149391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:33.151015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:33.152878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 148/289] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:32 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:33.160190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:33.180869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca71af-e27a-42d5-a888-11f370a45fa6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:33.245049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:33.245942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:33.246230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:33.246509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] PATH_INFO: `/v3/users/d793c666354a42aa8fb101239a4d368a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:33.247793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] RBAC: Authorizing `identity:get_user(user_id=d793c666354a42aa8fb101239a4d368a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:33.249261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:33.252334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 142/290] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:33 2022] GET /identity/v3/users/d793c666354a42aa8fb101239a4d368a => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:33.257690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:33.313881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:33.371127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad952b-c3f0-41d4-b4be-b741f61af013 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:33.422182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:33.423127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:33.423415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:33.423694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/38ed7222c0c34029988f44df93b8262c/roles/d36c3084b96844ca9af694880d44ac15` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:33.431675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef78dcb-c059-4fc0-9cd5-68bda5a35699 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:33.431965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] RBAC: Authorizing `identity:create_grant(project_id=9a2df333a2bd4f919ac91e69ef586e3c, user_id=38ed7222c0c34029988f44df93b8262c, role_id=d36c3084b96844ca9af694880d44ac15)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:33.433425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:33.461208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 149/291] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:51:33 2022] PUT /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/users/38ed7222c0c34029988f44df93b8262c/roles/d36c3084b96844ca9af694880d44ac15 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:33.468980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:33.469646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:33.469917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:33.470198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:33.548548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:33.550084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:33.593976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:33.676875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 143/292] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:51:33.682163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:33.713561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:33.791164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d60b22-dadf-4eaf-a399-ea5089a1be7f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:33.835343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec409-85c5-4a1a-9af8-9beb49b88763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:33.836265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec409-85c5-4a1a-9af8-9beb49b88763 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:33.836557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec409-85c5-4a1a-9af8-9beb49b88763 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:33.836839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec409-85c5-4a1a-9af8-9beb49b88763 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:33.838194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec409-85c5-4a1a-9af8-9beb49b88763 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:33.839672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec409-85c5-4a1a-9af8-9beb49b88763 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:33.841630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 150/293] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:33 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:33.847180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:33.969003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea81bd3-d12d-4202-b72a-3e601a280896 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:34.001819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:34.013393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:34.014066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:34.014356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:34.014649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:34.015513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=d793c666354a42aa8fb101239a4d368a) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:34.015829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:34.017281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36964b63-5caa-4dcc-8baf-2895ed3a0d1e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:34.025663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 144/294] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:33 2022] GET /identity/v3/role_assignments?user.id=d793c666354a42aa8fb101239a4d368a&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=8182238074534b76a65d54c2edbeffad => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:34.903960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cbe0d-b6c1-4126-98b1-9726c0f8cf21 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:34.904992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cbe0d-b6c1-4126-98b1-9726c0f8cf21 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:34.905486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cbe0d-b6c1-4126-98b1-9726c0f8cf21 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:34.907409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 151/295] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:34.913168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad5e1b-6fca-4170-99e2-010f0e206704 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:34.914142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad5e1b-6fca-4170-99e2-010f0e206704 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:34.915257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad5e1b-6fca-4170-99e2-010f0e206704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:34.932925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b4ad5e1b-6fca-4170-99e2-010f0e206704 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:34.934436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b4ad5e1b-6fca-4170-99e2-010f0e206704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:35.002973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 145/296] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:35.027246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cbe0d-b6c1-4126-98b1-9726c0f8cf21 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.126376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.134077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cbe0d-b6c1-4126-98b1-9726c0f8cf21 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.169647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.170829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.171316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.171787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.182650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:35.190770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7138c6-9541-4e62-9d26-b9c0ebc43167 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:35.191031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:35.191874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:35.192640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:35.193239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7138c6-9541-4e62-9d26-b9c0ebc43167 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.193239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7138c6-9541-4e62-9d26-b9c0ebc43167 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.193239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 146/297] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:35.195398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 152/298] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:35.200803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bd3f3-4ac2-4df7-a668-b66a02037b1f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.200803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bd3f3-4ac2-4df7-a668-b66a02037b1f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.200803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bd3f3-4ac2-4df7-a668-b66a02037b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.202012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.216028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-886bd3f3-4ac2-4df7-a668-b66a02037b1f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:35.217272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-886bd3f3-4ac2-4df7-a668-b66a02037b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:35.295896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 147/299] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:35.297302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b2660-9724-497c-8726-d198c473f07d None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.297596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b2660-9724-497c-8726-d198c473f07d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.297878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b2660-9724-497c-8726-d198c473f07d None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.299329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 148/300] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:35.304684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.305518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.305803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.316942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816be8c-52d2-451c-92f6-2aa97d47bd67 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.322813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:35.323997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:35.362319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.363101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.363490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.363840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.364603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:35.364915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:35.366297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:35.375893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 153/301] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:35.375893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.381133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.398418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.400750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 149/302] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:35.403529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.494757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ed21d-3c34-4552-bf82-6f9bde3b1340 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.515948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaab7ff-db92-467a-b654-7e2f684f985a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.531122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.531921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.532208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.532490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.533884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:35.535159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:35.536927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 154/303] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:35.537545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.557458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.558342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.558656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.558937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] PATH_INFO: `/v3/users/a593f4a721114675b592d947549366f4` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.573743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a593f4a721114675b592d947549366f4)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:35.575254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:35.577103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 150/304] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/users/a593f4a721114675b592d947549366f4 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:35.577678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.624543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.638025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.655199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e24894-e6a9-4b8f-ab0d-fc30a700522f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.682419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769e897-cc24-4d5d-abee-c5fd4a50a0f5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.690816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.691663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.691957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.692231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.704146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:35.713479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:35.714427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:35.715252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 155/305] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:35.715903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.724944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.726405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.727625 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.728515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] PATH_INFO: `/v3/groups` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.729776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:35.731696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:35.758832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 151/306] 10.222.0.39 () {60 vars in 1134 bytes} [Wed Jul 13 20:51:35 2022] POST /identity/v3/groups => generated 229 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:35.758832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:35.845676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa95b6-5a78-45a7-9ec3-f153c10b9198 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.882894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846dda37-da5d-4a13-b2a3-6a3f4707b969 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:35.884821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.885490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.885775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.886054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.886318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.886952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] RBAC: Adding query filter params (role.id=d36c3084b96844ca9af694880d44ac15, user.id=a593f4a721114675b592d947549366f4, scope.system=all) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:35.887259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:35.888786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c746bff-654d-4ebb-9aea-7a64c26b2652 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:35.894709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 156/307] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/role_assignments?user.id=a593f4a721114675b592d947549366f4&scope.system=all&role.id=d36c3084b96844ca9af694880d44ac15 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:35.896038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e99f-4828-40d6-813a-e49c0f8d68bd None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.896325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e99f-4828-40d6-813a-e49c0f8d68bd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.896601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6e99f-4828-40d6-813a-e49c0f8d68bd None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.897963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 157/308] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:35.904692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.905810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.907138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.929465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:35.930704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:35.935463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:35.947357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:35.948206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:35.948562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:35.948931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:35.949778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:35.950151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:35.952453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:35.961832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 152/309] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:35.967450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.034407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 158/310] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:36.062543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.112245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53c91d-981f-4a7b-a653-3d235c1171bd None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.148179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.149591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.149872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.150148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] PATH_INFO: `/v3/users/3cbfab52e5ba4b19861aab6a9eb5be80` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.151362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] RBAC: Authorizing `identity:get_user(user_id=3cbfab52e5ba4b19861aab6a9eb5be80)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.152809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.154748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 153/311] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:35 2022] GET /identity/v3/users/3cbfab52e5ba4b19861aab6a9eb5be80 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:36.160645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.172122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3e69b-ddad-434e-9066-aee13617b682 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.207818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.210862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.212566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.213027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.213495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.223347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.227600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:36.236710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:36.237605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:36.238400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:36.245911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 159/312] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:36.247736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.288909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad442d2-f7bb-4e57-9113-fb51c0e1c8dd None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.335869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.337328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.337620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.337931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] PATH_INFO: `/v3/system/users/3cbfab52e5ba4b19861aab6a9eb5be80/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.339483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3cbfab52e5ba4b19861aab6a9eb5be80, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.340921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.362655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 154/313] 10.222.0.39 () {58 vars in 1334 bytes} [Wed Jul 13 20:51:36 2022] PUT /identity/v3/system/users/3cbfab52e5ba4b19861aab6a9eb5be80/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:36.378213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483cab2-bfb6-4653-810a-bda88f281273 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.419256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.419940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.420227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.420503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.421274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:36.421746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.423303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.434147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 160/314] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:36.439798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.551007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.555463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099291fe-d86f-4308-93c0-b721550cf871 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.592945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.593828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.594107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.594381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] PATH_INFO: `/v3/users/38ed7222c0c34029988f44df93b8262c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.595663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] RBAC: Authorizing `identity:get_user(user_id=38ed7222c0c34029988f44df93b8262c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.597149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.599012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 155/315] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:36 2022] GET /identity/v3/users/38ed7222c0c34029988f44df93b8262c => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:36.607680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:36.674442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.716927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca844db-2f65-4e8d-91b9-c70223ca45e1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:36.755948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:36.756603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:36.756889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:36.757171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:36.828698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:36.830077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:36.892625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:36.934553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 161/316] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:36.942487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.053288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d8078-3e61-4d39-98df-1159b789be81 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.089937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b13009-ce7a-4771-a0af-bd96e790c9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.091157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b13009-ce7a-4771-a0af-bd96e790c9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.091645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b13009-ce7a-4771-a0af-bd96e790c9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.092161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b13009-ce7a-4771-a0af-bd96e790c9b3 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.093585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b13009-ce7a-4771-a0af-bd96e790c9b3 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.095278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b13009-ce7a-4771-a0af-bd96e790c9b3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.097534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 156/317] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:36 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:37.104653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:37.207806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b65f4e-0464-4158-96a4-3e653f0ed5f6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:37.242318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:37.243497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.243783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.244059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.244887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] RBAC: Adding query filter params (role.id=d36c3084b96844ca9af694880d44ac15, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c, user.id=38ed7222c0c34029988f44df93b8262c) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:37.245200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:37.246647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7dd0b-c5d6-4447-ab16-a4699c60afd2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:37.252884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 162/318] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:51:37 2022] GET /identity/v3/role_assignments?user.id=38ed7222c0c34029988f44df93b8262c&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=d36c3084b96844ca9af694880d44ac15 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:37.933160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf10be-94b3-4f2f-9a9e-3dadc5ca90e4 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.933452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf10be-94b3-4f2f-9a9e-3dadc5ca90e4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.933735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf10be-94b3-4f2f-9a9e-3dadc5ca90e4 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.935188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 157/319] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:37.935588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa8cfa-03b2-42da-8e69-da5dd8f44bb6 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.936108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa8cfa-03b2-42da-8e69-da5dd8f44bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.936272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa8cfa-03b2-42da-8e69-da5dd8f44bb6 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.937581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 163/320] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:37.940172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.940463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.940746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.942832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:37.943311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:37.943769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:37.959769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:37.960577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:37.961014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:37.962283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:37.984247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:37.994053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.037430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 158/321] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:38.042884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 164/322] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:38.059357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.059598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.163527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b31d7-3437-4b45-a47d-0d8f99d2efa3 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.166357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac43741-0d2f-4073-a011-cfaef768f4e1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.198838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.199548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.199834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.200112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.203738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.204536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.204821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.205101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.206277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.207769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.209551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 159/323] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:38.211083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:38.215306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:38.219394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:38.220244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:38.221010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 165/324] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:38.226128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.244665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.252909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.332202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc7e62-9628-45f0-a060-651fcd9a2e77 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.341639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aceba-ba70-4acd-8bc2-bd6529e85c5f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.370267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c82417-70ae-4dea-a84a-8f63b61063b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.371241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c82417-70ae-4dea-a84a-8f63b61063b9 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.371670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c82417-70ae-4dea-a84a-8f63b61063b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.372090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c82417-70ae-4dea-a84a-8f63b61063b9 None admin] PATH_INFO: `/v3/groups` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.372941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c82417-70ae-4dea-a84a-8f63b61063b9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.374648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c82417-70ae-4dea-a84a-8f63b61063b9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.377111 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.377846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.378203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.378548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.379588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:38.379851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.381210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.390704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 166/325] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:38.392607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 160/326] 10.222.0.39 () {60 vars in 1134 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:38.398743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.413635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb0614-5fcf-4610-beb8-b7010e69811b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.413981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb0614-5fcf-4610-beb8-b7010e69811b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.414778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb0614-5fcf-4610-beb8-b7010e69811b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.416526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 161/327] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:38.422030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.422334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.422601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.440727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:38.441970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:38.496175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.517019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 162/328] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:38.522582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d2791-dfe6-44a0-919a-77ecb136af61 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.540231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.561171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.562055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.562339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.562667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] PATH_INFO: `/v3/users/a593f4a721114675b592d947549366f4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.565548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] RBAC: Authorizing `identity:get_user(user_id=a593f4a721114675b592d947549366f4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.566178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.567814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.567936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 167/329] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/users/a593f4a721114675b592d947549366f4 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:38.575293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.663469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655eb477-9b5c-49c7-b913-07b6ff438735 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.700943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd32b-8845-40c0-b80a-7dbd112f4a7a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.711391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.712194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.712477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.712750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.729626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:38.738017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:38.738957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:38.739753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 163/330] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:38.746089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.746734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.747534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.747819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.748131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] PATH_INFO: `/v3/system/users/a593f4a721114675b592d947549366f4/roles/d36c3084b96844ca9af694880d44ac15` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.750824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a593f4a721114675b592d947549366f4, role_id=d36c3084b96844ca9af694880d44ac15)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.752333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.763531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 168/331] 10.222.0.39 () {58 vars in 1334 bytes} [Wed Jul 13 20:51:38 2022] PUT /identity/v3/system/users/a593f4a721114675b592d947549366f4/roles/d36c3084b96844ca9af694880d44ac15 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:38.848423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:38.871847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b1535-4455-4e6f-a960-5a2a49ca8aca None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:38.909009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:38.909699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:38.909986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:38.910262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:38.911074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:38.911383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:38.912818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:38.923159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 164/332] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:38.928756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:38.942579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:39.037889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51284a3-d0f3-4590-b4ae-72be0d4664e6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:39.076259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da3d8b-ef77-47c7-917d-bf449357f054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:39.077195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3d8b-ef77-47c7-917d-bf449357f054 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.077485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3d8b-ef77-47c7-917d-bf449357f054 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.077763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da3d8b-ef77-47c7-917d-bf449357f054 None admin] PATH_INFO: `/v3/users/3cbfab52e5ba4b19861aab6a9eb5be80` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.079000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da3d8b-ef77-47c7-917d-bf449357f054 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbfab52e5ba4b19861aab6a9eb5be80)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:39.080443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da3d8b-ef77-47c7-917d-bf449357f054 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:39.082381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 169/333] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:38 2022] GET /identity/v3/users/3cbfab52e5ba4b19861aab6a9eb5be80 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:39.088525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:39.203678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de90f8-fb2e-46af-8a45-f06e8d13e7f8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:39.242819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:39.243482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:39.243770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:39.244049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:39.244898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] RBAC: Adding query filter params (role.id=8182238074534b76a65d54c2edbeffad, user.id=3cbfab52e5ba4b19861aab6a9eb5be80, scope.system=all) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:39.245210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:39.249115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080393a-1f5f-4f11-a53b-ded94a980a93 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:39.264725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 165/334] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Jul 13 20:51:39 2022] GET /identity/v3/role_assignments?user.id=3cbfab52e5ba4b19861aab6a9eb5be80&scope.system=all&role.id=8182238074534b76a65d54c2edbeffad => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:40.772428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71636c78-8938-46ff-8c34-58cdfb6a319c None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:40.772927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71636c78-8938-46ff-8c34-58cdfb6a319c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:40.773001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71636c78-8938-46ff-8c34-58cdfb6a319c None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:40.775044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 170/335] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:40.780771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804c029-7cbf-4812-bb46-493a3ce8f3c7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:40.781639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804c029-7cbf-4812-bb46-493a3ce8f3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:40.782118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804c029-7cbf-4812-bb46-493a3ce8f3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:40.787832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:40.807867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9804c029-7cbf-4812-bb46-493a3ce8f3c7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:40.809105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9804c029-7cbf-4812-bb46-493a3ce8f3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:40.892616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 166/336] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:40.928148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71636c78-8938-46ff-8c34-58cdfb6a319c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:40.939213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721b94b-31f2-4e2c-b706-c6dcf025dea1 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:40.939489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721b94b-31f2-4e2c-b706-c6dcf025dea1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:40.939765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721b94b-31f2-4e2c-b706-c6dcf025dea1 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:40.941282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 167/337] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:40.946938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:40.947419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:40.947840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:40.971562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:40.973770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:41.070138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.103754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 168/338] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:41.111493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71636c78-8938-46ff-8c34-58cdfb6a319c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.156760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.157583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.157867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.158147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.163147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:41.176327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:41.184379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:41.185246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:41.186042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:41.191215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 171/339] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:41.193808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:41.263100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.336691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869006de-3e34-4f6c-9b82-a57f9a5098d6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.338980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e11d8a-aaf1-4638-af1a-e9945c9b42cf None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.378383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.379105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.379390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.379707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.380636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:41.381092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:41.382711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7722826-2655-426c-a69a-410c88d47809 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:41.387848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301a15a-c9e1-46c3-a328-1c12f6ed7d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.388657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301a15a-c9e1-46c3-a328-1c12f6ed7d7e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.388938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301a15a-c9e1-46c3-a328-1c12f6ed7d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.389213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301a15a-c9e1-46c3-a328-1c12f6ed7d7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.389577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.394889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 172/340] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:41.396341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d5c7c-54ee-4415-a99d-7fd1809e4323 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.396777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d5c7c-54ee-4415-a99d-7fd1809e4323 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.397221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d5c7c-54ee-4415-a99d-7fd1809e4323 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.402811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301a15a-c9e1-46c3-a328-1c12f6ed7d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:41.403912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 173/341] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:41.405430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d5c7c-54ee-4415-a99d-7fd1809e4323 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-0301a15a-c9e1-46c3-a328-1c12f6ed7d7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:41.419364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:41.420461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:41.421415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:41.423368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 169/342] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:41.424741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e77898-69e6-44c4-b911-30528494088f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.425227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e77898-69e6-44c4-b911-30528494088f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.425671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e77898-69e6-44c4-b911-30528494088f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.458642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-26e77898-69e6-44c4-b911-30528494088f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:41.460053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-26e77898-69e6-44c4-b911-30528494088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:41.519831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d5c7c-54ee-4415-a99d-7fd1809e4323 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.557870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.558900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.559818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.560280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.590220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 170/343] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:41.591534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e77898-69e6-44c4-b911-30528494088f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:41.648192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.652167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:41.654757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:41.659008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.716374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e77898-69e6-44c4-b911-30528494088f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.760958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.761978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.762473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.762977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.763872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:41.764338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:41.765959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:41.776078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 171/344] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:41.777149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:41.784035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 174/345] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:51:41.785750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:41.904322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.909418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352b99e-a660-4410-90ac-43e056f56704 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.911862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e8e7-6806-432f-b874-fa1d45cfc970 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:41.935590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:41.956486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.957268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.957891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.958693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.964009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:41.965010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:41.965442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:41.965864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:41.982146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:41.991442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:41.993215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 20:51:41.994011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:41.996877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 172/346] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:41.998075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.037204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.038816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.109292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121cdd7c-74ad-4b41-9f5b-5b7a9f8ee152 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.142362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:42.146716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 175/347] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:42.147613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.149695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.150821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.151297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.151762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.153535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.155445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.157415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 173/348] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:41 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.158372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.219155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:42.270775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd075b-9d7c-4ab2-a7d1-3f9a80ca7515 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.307616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.308214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.308565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.308909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.310179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.311748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.313766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 174/349] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.314601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.350596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05e88f-0be2-462f-92f7-747a69e79104 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.409093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.409774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.410002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.410275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.411058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:42.411362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.412732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.429655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa04c29-c03a-4df3-9b0c-98311643cdee None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.430437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 176/350] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.431308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.442992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:42.475879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.477486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.477772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.478259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.494496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.496050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.500334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 175/351] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.501148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.538354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:42.602228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817b0e6-50b6-41e9-957a-ff5acdff263a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.639484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93f5a4-e775-48f4-aa13-a9c55c415ba2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.642557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.643591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.643973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.644350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.645552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.647033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.649316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 177/352] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.650086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.677355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77c7ca-e81e-43b0-88e7-6052b806ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.678445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77c7ca-e81e-43b0-88e7-6052b806ce88 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.678758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77c7ca-e81e-43b0-88e7-6052b806ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.679044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77c7ca-e81e-43b0-88e7-6052b806ce88 None admin] PATH_INFO: `/v3/users/a593f4a721114675b592d947549366f4` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.681492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77c7ca-e81e-43b0-88e7-6052b806ce88 None admin] RBAC: Authorizing `identity:get_user(user_id=a593f4a721114675b592d947549366f4)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.683136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77c7ca-e81e-43b0-88e7-6052b806ce88 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.684834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 176/353] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/users/a593f4a721114675b592d947549366f4 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.686375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0504f7-2b48-429c-83d4-96078ccffaa1 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.686683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0504f7-2b48-429c-83d4-96078ccffaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.686968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0504f7-2b48-429c-83d4-96078ccffaa1 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.688390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 177/354] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:42.688930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0504f7-2b48-429c-83d4-96078ccffaa1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.743021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:42.757322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679feb4e-42a7-4c8f-8393-aa8466dcc167 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.788612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:42.795268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.796132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.796559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.796880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.797814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=95b15952550141d9b3f8fec2905c4400) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:42.798244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.799809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.806106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 178/355] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=8182238074534b76a65d54c2edbeffad => generated 260 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.807018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:42.809142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0504f7-2b48-429c-83d4-96078ccffaa1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.850267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:42.850972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.851296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.851635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.852488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=95b15952550141d9b3f8fec2905c4400) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:42.852806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:42.854525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f5dc6-aa5c-4e0d-b387-833c66daceba None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:42.861594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 178/356] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:42.862947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:42.863240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:42.863572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:42.884051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:42.885264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:42.951992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9634aa5-52b1-4f20-a08a-db1f00f56878 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:42.969298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 179/357] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:43.010036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:43.010849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.011200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.011545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.012472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] RBAC: Adding query filter params (role.id=d36c3084b96844ca9af694880d44ac15, user.id=a593f4a721114675b592d947549366f4, scope.system=all) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:43.012852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:43.014532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120d189-6816-44ad-88d4-0ca7acea2af0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:43.032889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 179/358] 10.222.0.39 () {56 vars in 1308 bytes} [Wed Jul 13 20:51:42 2022] GET /identity/v3/role_assignments?user.id=a593f4a721114675b592d947549366f4&scope.system=all&role.id=d36c3084b96844ca9af694880d44ac15 => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:43.042462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:43.086335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:43.162133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84534e-e641-41d6-af6d-97d65e983e38 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:43.186594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb7342-16a9-443e-a103-b561646a31e1 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.186953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb7342-16a9-443e-a103-b561646a31e1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.187215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb7342-16a9-443e-a103-b561646a31e1 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.188600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 180/359] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:43.193909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc697e-3a50-482d-9778-974186c02190 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.194189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc697e-3a50-482d-9778-974186c02190 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.194474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc697e-3a50-482d-9778-974186c02190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.216876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:43.218043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.218531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.219067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.226279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-87fc697e-3a50-482d-9778-974186c02190 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:43.230721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-87fc697e-3a50-482d-9778-974186c02190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:43.238324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:43.248519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:43.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:43.250331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 180/360] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:43.255933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:43.332222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:43.357380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 181/361] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:43.386960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc697e-3a50-482d-9778-974186c02190 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:43.459683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:43.468816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e388e0e-2171-426b-ba9b-0e96194fea2d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:43.517444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:43.518128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.518420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.518757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.519527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:43.519835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:43.521229 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b2727-a67f-4400-bd40-9db66e7989be None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:43.532511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 181/362] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:43.534606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025eb0f-b592-43bb-b16e-71ae81f605ec None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.535111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025eb0f-b592-43bb-b16e-71ae81f605ec None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.535558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025eb0f-b592-43bb-b16e-71ae81f605ec None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.537755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 182/363] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:43.538804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025eb0f-b592-43bb-b16e-71ae81f605ec None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:43.567083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc697e-3a50-482d-9778-974186c02190 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:43.614205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf6f24-8b78-4119-af0c-e8a39e32080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:43.615539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf6f24-8b78-4119-af0c-e8a39e32080d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.616378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf6f24-8b78-4119-af0c-e8a39e32080d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.616795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf6f24-8b78-4119-af0c-e8a39e32080d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.635640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf6f24-8b78-4119-af0c-e8a39e32080d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-28bf6f24-8b78-4119-af0c-e8a39e32080d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:43.646465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:43.647412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:43.648190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 182/364] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:43.649389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.649389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.650329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.675244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:43.676638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:43.683456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025eb0f-b592-43bb-b16e-71ae81f605ec None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:43.734548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:43.735833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.736396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:43.736695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.737348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.773849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 183/365] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:43.775138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:43.801675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:43.852465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:43.853925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:43.904308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d470f-11f2-411c-9582-77180dcf8290 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:43.944445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:43.945665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:43.945954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:43.946232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:43.947021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:43.947327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:43.948820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:43.958656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 184/366] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:43.959575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:43.980822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 183/367] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:43.982115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.002775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.058583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.079905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725295a2-3696-4a4d-b4ae-b299d5fd4f31 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.108309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98e29a-bfec-4ea4-96f5-2b7e8b29f359 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.116615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.117457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.117746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.118026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.129209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:44.140062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:44.140923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:44.141702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 185/368] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:44.142843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.149963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.150650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.150946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.151222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.224038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.225532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.255860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938977da-24a0-4b79-abbc-37e4b8ab6967 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.258901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.295623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.296500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.296888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.297169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.298477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.299857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.300753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.306992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 186/369] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:44.308045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.345551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 184/370] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:44.346524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.424581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde2271-9e97-44a7-96f3-bb4a597a8247 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.462053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.462761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.463044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.463319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.464078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:44.464386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.465770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.469505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6559797-e4ab-4b40-a427-977dfc4bf74f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.476342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 187/371] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:44.477154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.503266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.521808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.522926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.523251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.523532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.526605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.528251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.531029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 185/372] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:44.531804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.567077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.590281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95dbf9-ed6b-4eb0-b068-c9efd1ebcd99 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.626799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.627771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.628055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.628335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.629533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.631018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.633604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 188/373] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:44.634475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.650653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9288b-b5f6-479d-9645-c76224a68982 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.695349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.696010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.696294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.696569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.771287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.783961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6b632-ee73-47f4-b0be-f64dc9f8b8e1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:44.791826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.794172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.826672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:44.827172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:44.827456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:44.827792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:44.828635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:44.828947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:44.830391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:44.834954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:44.836955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 189/374] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=8182238074534b76a65d54c2edbeffad => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:44.838116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.934721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 186/375] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:44.942035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:44.973527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feffaab-16bd-461a-98c2-76934623838d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.019642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.020557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.020883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.021162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.022397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.024445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.026305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 190/376] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:45.032818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.063965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.077010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74688d-0f2f-42e6-b39c-68864483b4aa None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.116682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.123381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.124730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.125247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.125864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] PATH_INFO: `/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.127474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] RBAC: Authorizing `identity:get_project(project_id=c113e9e1592248a9b12fd4d2b385aa5a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.129356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.134875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 187/377] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:44 2022] GET /identity/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:45.137557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.149662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64e4ea-8332-4de6-8c96-c05dbe1f80d0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.190498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.191365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.191714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.192106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.193073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:45.193449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.195244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.201808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 191/378] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:45.267223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5768ff-b7dc-4050-a886-91e1fb451179 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.318430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117d5fc-02af-42c2-ab9a-8bd06c135ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.319268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117d5fc-02af-42c2-ab9a-8bd06c135ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.319552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117d5fc-02af-42c2-ab9a-8bd06c135ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.319827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117d5fc-02af-42c2-ab9a-8bd06c135ee6 None admin] PATH_INFO: `/v3/groups/3d214cbbea2a4a31bab7fef576a7f690` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.326068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117d5fc-02af-42c2-ab9a-8bd06c135ee6 None admin] RBAC: Authorizing `identity:get_group(group_id=3d214cbbea2a4a31bab7fef576a7f690)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.327642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d117d5fc-02af-42c2-ab9a-8bd06c135ee6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.329715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 188/379] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/groups/3d214cbbea2a4a31bab7fef576a7f690 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:45.335457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.357160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c47a8-8922-4865-9a45-741dc9887836 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.357466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c47a8-8922-4865-9a45-741dc9887836 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.357723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c47a8-8922-4865-9a45-741dc9887836 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.359338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 189/380] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:45.372546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.372869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.373111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.399661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:45.401751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:45.472952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.474264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.481188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 190/381] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:45.504882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb426f-3a36-4460-8e8f-804529f8d603 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.506961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.546936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.547934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.548436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.548852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.549878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] RBAC: Adding query filter params (group.id=3d214cbbea2a4a31bab7fef576a7f690, role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c113e9e1592248a9b12fd4d2b385aa5a) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:45.550422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.552022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.558901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 192/382] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/role_assignments?group.id=3d214cbbea2a4a31bab7fef576a7f690&scope.project.id=c113e9e1592248a9b12fd4d2b385aa5a&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:45.633650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f0d21-07c6-44c3-b1f1-494de3236640 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.672035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b5806-a813-4f2b-9539-be1c4e39f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.672858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5806-a813-4f2b-9539-be1c4e39f28e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.673142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5806-a813-4f2b-9539-be1c4e39f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.673422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b5806-a813-4f2b-9539-be1c4e39f28e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.690862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b5806-a813-4f2b-9539-be1c4e39f28e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-860b5806-a813-4f2b-9539-be1c4e39f28e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:45.700970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:45.701899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:45.703121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:45.705710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 191/383] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:45.712565 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.792326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ae04d-0939-48ae-b09a-fc84062c415b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.792944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ae04d-0939-48ae-b09a-fc84062c415b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.793672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ae04d-0939-48ae-b09a-fc84062c415b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.796436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 192/384] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:45.803823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.804886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.805706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.828843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:45.830669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:45.834830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcae5ee-b9f1-44f8-807c-3312f920b199 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:45.887117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:45.888217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:45.888703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:45.889216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:45.890190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:45.891038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:45.892634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:45.898953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.910604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 193/385] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:45.926859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:45.953137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:45.955428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 193/386] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:46.026174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.088660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e1f40-2095-4da4-b281-c2eca823d0ed None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.160872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.162089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.162573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.163083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.167073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.168778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.171881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 194/387] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:45 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:46.181113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.227878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefa3ab-4e28-4bb8-bb5b-8a02f0c36340 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.286035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:46.309519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249228d-3de6-4c02-a5a3-73cff4008523 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.324131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.325090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.325470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.325845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.350646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:46.354490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:46.355469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:46.356289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:46.357355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.358458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 194/388] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:46.358697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.362298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.362608 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.364062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.365066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.366495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.376050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 195/389] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:46 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:46.381690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.437636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:46.531128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83b6d-f45c-4ec3-8e70-30fe810783c2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.572379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.573074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.573364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.573645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.574418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:46.574749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.576150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.589355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf18da4-fe26-4448-ad5f-4b2dd264b89a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.601090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 195/390] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:46.607682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.638210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.639187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.639471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.639753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400/groups/2417a4a63c334b8d983365c0b942d028/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.642036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=95b15952550141d9b3f8fec2905c4400, group_id=2417a4a63c334b8d983365c0b942d028, role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.643748 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.662268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 196/391] 10.222.0.39 () {58 vars in 1442 bytes} [Wed Jul 13 20:51:46 2022] PUT /identity/v3/projects/95b15952550141d9b3f8fec2905c4400/groups/2417a4a63c334b8d983365c0b942d028/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:51:46.802319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:46.805852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb7b22-3083-4c91-853b-0e1a8e6548bc None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:46.853521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:46.854878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:46.855381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:46.855856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:46.861451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:46.863220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:46.865447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 196/392] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:46 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:46.870667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:46.890453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:46.988157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccba0cb-274b-4df7-aa61-aa2e520c28d9 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.030579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d203f-616c-4356-b33f-2ffb9ce400f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.031873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d203f-616c-4356-b33f-2ffb9ce400f1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.032392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d203f-616c-4356-b33f-2ffb9ce400f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.032885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d203f-616c-4356-b33f-2ffb9ce400f1 None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.034814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d203f-616c-4356-b33f-2ffb9ce400f1 None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.036478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d203f-616c-4356-b33f-2ffb9ce400f1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.039708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 197/393] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:46 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:47.044474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.167287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd354e-2236-4240-b8cf-5b853fe72319 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:47.209714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c501a-08cb-45e0-92d0-09103d15043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:47.211197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c501a-08cb-45e0-92d0-09103d15043e None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.211491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c501a-08cb-45e0-92d0-09103d15043e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.211772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c501a-08cb-45e0-92d0-09103d15043e None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400/groups/2417a4a63c334b8d983365c0b942d028/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.213769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c501a-08cb-45e0-92d0-09103d15043e None admin] RBAC: Authorizing `identity:create_grant(project_id=95b15952550141d9b3f8fec2905c4400, group_id=2417a4a63c334b8d983365c0b942d028, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:47.219350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c501a-08cb-45e0-92d0-09103d15043e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:47.246869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 197/394] 10.222.0.39 () {58 vars in 1442 bytes} [Wed Jul 13 20:51:47 2022] PUT /identity/v3/projects/95b15952550141d9b3f8fec2905c4400/groups/2417a4a63c334b8d983365c0b942d028/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:47.735969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1ec3f-b9c1-4e22-8aeb-4fedcfd47d66 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.736976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1ec3f-b9c1-4e22-8aeb-4fedcfd47d66 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.737468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1ec3f-b9c1-4e22-8aeb-4fedcfd47d66 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.739400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 198/395] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:47.745200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417795a4-4a5e-4134-9cb1-cb36dcc83bba None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:47.745489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417795a4-4a5e-4134-9cb1-cb36dcc83bba None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:47.745766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417795a4-4a5e-4134-9cb1-cb36dcc83bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:47.764079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-417795a4-4a5e-4134-9cb1-cb36dcc83bba None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:47.765325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-417795a4-4a5e-4134-9cb1-cb36dcc83bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:47.790386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.839868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 198/396] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:47.862832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1ec3f-b9c1-4e22-8aeb-4fedcfd47d66 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:47.953559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:47.984647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e1ec3f-b9c1-4e22-8aeb-4fedcfd47d66 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.007791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d92d0-344a-4a9f-b3ec-7e4b31d82d99 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.008079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d92d0-344a-4a9f-b3ec-7e4b31d82d99 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.008269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d92d0-344a-4a9f-b3ec-7e4b31d82d99 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.009921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 199/397] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:48.016269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae2f82-3f5b-4590-9dcf-b0cae048ce96 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.016560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae2f82-3f5b-4590-9dcf-b0cae048ce96 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.016843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae2f82-3f5b-4590-9dcf-b0cae048ce96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.029781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.030955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.031442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.031936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.048272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:48.057046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:48.057924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:48.058750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:48.059839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-fbae2f82-3f5b-4590-9dcf-b0cae048ce96 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:48.060875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fbae2f82-3f5b-4590-9dcf-b0cae048ce96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.061533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 199/398] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:48.068805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.189200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 200/399] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:48.228194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4241e-d7e9-4a47-929f-2c91f1b31ade None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.228531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4241e-d7e9-4a47-929f-2c91f1b31ade None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.228762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4241e-d7e9-4a47-929f-2c91f1b31ade None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.230400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 201/400] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:48.235442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.235728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.236003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.248111 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40be7b9-3d1b-43a0-97e6-da2796456aac None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.259510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:48.260922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:48.295187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.296929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.297222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.297502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.298651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:48.298962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.300799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.308834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.314128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 200/401] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:48.315163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.354493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 202/402] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:48.359470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.393356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.474641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace59cdf-bef7-475a-aa7f-4b80d15dbd7f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.522198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4891b-354f-4512-b3d7-74eb80006852 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.526873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.527682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.527968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.528246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.542506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:48.552376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:48.553288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:48.554062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:48.555868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 201/403] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:48.558678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.570653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.571972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.572257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.572868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.574390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.576730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.579311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 203/404] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:48.580136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.634537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.699334 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e018e8-4f71-433e-9c56-a0ef1679bec3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.705402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.735490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6e84f-8f9b-4150-8ef2-9e3c215f5e57 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.750788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.751971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.752264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.752545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.766252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:48.774677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:48.775547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:48.776315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 202/405] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:48.779250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.798774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.799515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.799846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.800124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.800905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] RBAC: Adding query filter params (name=member) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:48.801199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.802654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.813920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 204/406] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:48.819307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.916116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d7ded-84e2-4f53-8c6c-2c6c9e31fc01 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.944013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:48.944503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf41460-04bf-4df1-8a1e-f62e63a97eee None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:48.962945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.963944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.964235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.964506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.965820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:48.967301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:48.969254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 203/407] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:48.969993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:48.993120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:48.994380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:48.995602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:48.996083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:48.997438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:48.997947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.000121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.016544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 205/408] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:49.017686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.031103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.097660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e5157-88b6-423f-a296-b3daca048783 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.125481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b118bd-38e5-4924-8fa2-2f4a8f24511b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.136633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.137497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.137778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.138053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.139762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.141347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.143331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 204/409] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:48 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:49.144175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.173373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.174637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.175125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.175656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/groups/2417a4a63c334b8d983365c0b942d028/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.179016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a2df333a2bd4f919ac91e69ef586e3c, group_id=2417a4a63c334b8d983365c0b942d028, role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.180715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.203432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 206/410] 10.222.0.39 () {58 vars in 1442 bytes} [Wed Jul 13 20:51:49 2022] PUT /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/groups/2417a4a63c334b8d983365c0b942d028/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:49.207210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.218803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.278377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e6c78-a741-4fff-9a36-cdf6fe827af4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.311858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.319452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccf089-2cc0-4aea-b967-419cd368cd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.320983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccf089-2cc0-4aea-b967-419cd368cd4c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.321337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccf089-2cc0-4aea-b967-419cd368cd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.321710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccf089-2cc0-4aea-b967-419cd368cd4c None admin] PATH_INFO: `/v3/groups/3d214cbbea2a4a31bab7fef576a7f690` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.323398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccf089-2cc0-4aea-b967-419cd368cd4c None admin] RBAC: Authorizing `identity:get_group(group_id=3d214cbbea2a4a31bab7fef576a7f690)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.324692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccf089-2cc0-4aea-b967-419cd368cd4c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.328475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 205/411] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/groups/3d214cbbea2a4a31bab7fef576a7f690 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:49.329973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da1972-0796-4cfc-94ae-a5648548bccc None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.334734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da1972-0796-4cfc-94ae-a5648548bccc None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.334734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da1972-0796-4cfc-94ae-a5648548bccc None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.334979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 206/412] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:49.338873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.339149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.339580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.351727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a53b8-0a9f-4cfc-84a1-86d333c90093 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.358926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:49.360234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:49.395739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.396972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.397515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.398595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.400707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.402448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.404442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 207/413] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:49.405451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.446737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 207/414] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:49.447778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.516024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b197d-3a6e-47b2-ae3c-070d38af52e1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.526992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.554954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.556339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.556825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.557275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] PATH_INFO: `/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.558798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] RBAC: Authorizing `identity:get_project(project_id=c113e9e1592248a9b12fd4d2b385aa5a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.560550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.562906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 208/415] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:49.563922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.574303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.579021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7d3d7-04c0-4003-b080-b87b8bfcf1cf None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.618352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.619686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.620206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.620670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/groups/2417a4a63c334b8d983365c0b942d028/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.623793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a2df333a2bd4f919ac91e69ef586e3c, group_id=2417a4a63c334b8d983365c0b942d028, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.625823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.642508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 208/416] 10.222.0.39 () {58 vars in 1442 bytes} [Wed Jul 13 20:51:49 2022] PUT /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c/groups/2417a4a63c334b8d983365c0b942d028/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:49.643727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.675184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de755a-e505-4b90-b706-1045d9bd33ae None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.714261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.715108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.715410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.715690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.727429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:49.735772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:49.736717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:49.737563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 209/417] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:49.743671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.768325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6985ff-27f0-4482-9eb8-1ab685b08621 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.785892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.810426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a5926-6d19-4344-81f2-4215629cfc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.812016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a5926-6d19-4344-81f2-4215629cfc57 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.812730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a5926-6d19-4344-81f2-4215629cfc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.813213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a5926-6d19-4344-81f2-4215629cfc57 None admin] PATH_INFO: `/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a/groups/3d214cbbea2a4a31bab7fef576a7f690/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.816329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a5926-6d19-4344-81f2-4215629cfc57 None admin] RBAC: Authorizing `identity:create_grant(project_id=c113e9e1592248a9b12fd4d2b385aa5a, group_id=3d214cbbea2a4a31bab7fef576a7f690, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.818927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a5926-6d19-4344-81f2-4215629cfc57 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.822216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:49.843413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 209/418] 10.222.0.39 () {58 vars in 1442 bytes} [Wed Jul 13 20:51:49 2022] PUT /identity/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a/groups/3d214cbbea2a4a31bab7fef576a7f690/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:51:49.879424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56eda6-890b-4eaa-875f-ecf0854fccaa None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:49.914988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb8719-4046-4275-8ab2-59a297230b15 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.915512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb8719-4046-4275-8ab2-59a297230b15 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.915933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb8719-4046-4275-8ab2-59a297230b15 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.917499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 210/419] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:51:49.923375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.923943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.924383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:49.924714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.925073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:49.925360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:49.925637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:49.926416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:49.926773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:49.928283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:49.939826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 210/420] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:49.947166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:49.950605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:49.952285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:50.051030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 211/421] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:50.079824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.107174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d3db-29a4-4750-a1cf-0541eab25af6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.154437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.172470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.173665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.174175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.174793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.198791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f70c69-497d-489d-be6e-fb4aad912111 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.201875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.259486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.260404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.260759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.261113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.293816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.295543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.296062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:50.316095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:50.317060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:50.317842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 212/422] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:50.327512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.409443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 211/423] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:50.416464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.466488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.536509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f4f00-1216-4a2d-88fa-a9e03632712e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.577396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd384-2e9e-44ed-94bb-d176d18828a8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.614191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.615534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.616047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.616500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.617923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.619596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.621824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 212/424] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:50.627833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.639481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.640452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.640918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.641370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.642374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:50.643477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.645071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.668175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 213/425] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:50.672207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:50.730575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.815779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505f1dd-f39a-43c8-87ac-7f1d9ec4b39d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.852342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:50.853259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:50.854020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:50.854648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:50.854952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:50.880600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9cf8-a644-483a-b3dd-19ad29a869bc None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:50.927883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:50.929806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:50.960255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.014163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.015378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.015865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.018195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.019625 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.022825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.028090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 213/426] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:51.032508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.060489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 214/427] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:51.065707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.181438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892011dc-47e2-42f0-90bd-b8cd97a53eee None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.219031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.223218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.224459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.224942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.225456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] PATH_INFO: `/v3/projects/95b15952550141d9b3f8fec2905c4400` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.227033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] RBAC: Authorizing `identity:get_project(project_id=95b15952550141d9b3f8fec2905c4400)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.230187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.239272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 215/428] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/projects/95b15952550141d9b3f8fec2905c4400 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:51.241076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c05229-45ef-413b-a34f-43dca67a933c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.244960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.355319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.355989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.356277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.356555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.357851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=95b15952550141d9b3f8fec2905c4400) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:51.358171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.359900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.372878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 214/429] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 611 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:51.397057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6d7bb-2495-41bc-b5b7-ac61436739aa None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.451593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62bf6b-3be3-4f67-8d3f-9ef1c5b64e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.452380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62bf6b-3be3-4f67-8d3f-9ef1c5b64e2a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.452666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62bf6b-3be3-4f67-8d3f-9ef1c5b64e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.452946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62bf6b-3be3-4f67-8d3f-9ef1c5b64e2a None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.459502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62bf6b-3be3-4f67-8d3f-9ef1c5b64e2a None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.461019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62bf6b-3be3-4f67-8d3f-9ef1c5b64e2a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.462973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 216/430] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:51.468698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.521434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.613966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2d5a-6583-4976-ba33-ca3d4c7e207a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.642451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae9fd-8d06-4ecc-a4bb-0148708eda5b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.642784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae9fd-8d06-4ecc-a4bb-0148708eda5b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.643038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ae9fd-8d06-4ecc-a4bb-0148708eda5b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.644414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 217/431] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:51.649592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.649868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.650151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.664999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.665664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.665949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.666225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.667095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=95b15952550141d9b3f8fec2905c4400) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:51.667407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:51.668295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:51.668845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:51.669477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:51.680148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 215/432] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=95b15952550141d9b3f8fec2905c4400&role.id=8182238074534b76a65d54c2edbeffad => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:51.751451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.757653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 218/433] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:51.783402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:51.887606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:51.897082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01089b55-98da-46bc-ab66-4b76c656ca2f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:51.934812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44f5fa-e62e-42c0-818f-2ea13eff974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:51.935987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44f5fa-e62e-42c0-818f-2ea13eff974e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:51.936471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44f5fa-e62e-42c0-818f-2ea13eff974e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:51.936942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44f5fa-e62e-42c0-818f-2ea13eff974e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:51.947911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44f5fa-e62e-42c0-818f-2ea13eff974e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-fa44f5fa-e62e-42c0-818f-2ea13eff974e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:51.956222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:51.957102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:51.957903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:51.960312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 216/434] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:51.966987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.076920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e343474-7efc-47ab-b57b-d1258c1f7886 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.113230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.114175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.114604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.115049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.115965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:52.116426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.117997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf5b1d-730f-41e4-9e0d-8c566b48ca9f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.125306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d20e0-a55e-411e-9ee5-e7364070c719 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.125620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d20e0-a55e-411e-9ee5-e7364070c719 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.125875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d20e0-a55e-411e-9ee5-e7364070c719 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.127112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 219/435] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:52.127512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 217/436] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:52.132720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9978184-a80f-46c9-9498-204f650f846c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.133010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9978184-a80f-46c9-9498-204f650f846c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.133110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d20e0-a55e-411e-9ee5-e7364070c719 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.133292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9978184-a80f-46c9-9498-204f650f846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.157521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a9978184-a80f-46c9-9498-204f650f846c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:52.158743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a9978184-a80f-46c9-9498-204f650f846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:52.218374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.238640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 220/437] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:52.239758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226ca1-ca94-4ca7-95f3-c67f989a6b1a None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.240040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226ca1-ca94-4ca7-95f3-c67f989a6b1a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.240314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226ca1-ca94-4ca7-95f3-c67f989a6b1a None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.241872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 221/438] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:52.247030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.247348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.247587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.266052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:52.267525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:52.272060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d20e0-a55e-411e-9ee5-e7364070c719 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.316639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.317293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.317576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.317876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.345638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.346855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 222/439] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:52.347687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.392460 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.393911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.453659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a4906-1158-4038-84d4-e4cfeff12e26 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.467509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.487510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.488293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.488574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.488847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.489867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 218/440] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:52.489867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.498849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:52.506426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:52.507868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:52.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 223/441] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:52.509307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.550458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.597605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5a4be-6a57-41f8-8d1f-34687430a7d0 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.605937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770ab30-fc1e-43eb-967a-7b347bfbeeb5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.629802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.630578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.630883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.631156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.638740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.639579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.639859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.640133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.641316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.642049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:52.642793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.644628 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 224/442] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:52.645299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:52.649684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:52.650567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:52.651605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 219/443] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:52.652641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.692563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.749326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07272227-3b45-4d37-b88d-f4918c1fb042 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.754330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca92e0-909b-45f9-b741-df3115115160 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.763648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.784140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.784680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.784808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.785082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.785360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.785518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.785800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.786075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.786168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:52.786406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.787271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.787832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.788695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.790554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 220/444] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:52.791320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.800865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 225/445] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:52.800865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.892787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190a9a2-4c0a-43ca-944d-804b02ddb6c5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.895016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b87d-9490-41ae-9385-2b7db5c3b4a3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:52.895162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:52.926634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.927290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.927579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.927856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.928431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:52.928690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:52.929003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.929153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:52.929901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:52.930180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:52.930957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:52.931057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.931265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:52.932620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:52.937026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 226/446] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:52.937797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.940659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 221/447] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:52.946012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:52.995685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.057509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2990a44-9bc8-49a5-bb5c-f506321d8f48 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.062434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7bc9-694b-4297-8a47-c8dc4af91bbb None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.098974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.099633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.099743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.099936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.100196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.100289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.100561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.100846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.151278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.178401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.180459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.185694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.187198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.220677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.278462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 222/448] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 20:51:53.288161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 227/449] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:51:53.288315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.290080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.353699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.380402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e20a9f-9a0e-40c4-9ef3-e09e180b567f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.388091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7f95-af5f-4b55-bd0e-53e86e4c007a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.414824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.415614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.415895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.416163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] PATH_INFO: `/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.417467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] RBAC: Authorizing `identity:get_project(project_id=c113e9e1592248a9b12fd4d2b385aa5a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.418928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.420620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 223/450] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/projects/c113e9e1592248a9b12fd4d2b385aa5a => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:53.422221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.423115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.423403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.423682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] PATH_INFO: `/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.424859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] RBAC: Authorizing `identity:get_project(project_id=9a2df333a2bd4f919ac91e69ef586e3c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.426085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.426281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.430808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 228/451] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/projects/9a2df333a2bd4f919ac91e69ef586e3c => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:53.433271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.439964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.528896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099724-5365-4291-a03b-85ccbc530cac None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.529940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19064861-7f3f-4fa8-a834-b9b176f566f7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.562273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.562490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.563146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.563359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.563447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.563642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.563727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] PATH_INFO: `/v3/groups/3d214cbbea2a4a31bab7fef576a7f690` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.563917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] PATH_INFO: `/v3/groups/2417a4a63c334b8d983365c0b942d028` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.564868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] RBAC: Authorizing `identity:get_group(group_id=3d214cbbea2a4a31bab7fef576a7f690)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.565085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] RBAC: Authorizing `identity:get_group(group_id=2417a4a63c334b8d983365c0b942d028)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.566318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.566498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.567968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 224/452] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/groups/3d214cbbea2a4a31bab7fef576a7f690 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:53.568371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 229/453] 10.222.0.39 () {56 vars in 1180 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/groups/2417a4a63c334b8d983365c0b942d028 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:53.573561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.573914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:53.577361 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.651455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:53.672537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11fd66-48c7-49fc-b785-192ae86e4951 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.673523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01055c5-b4c3-4bd9-bca9-f3f47d0f9b2d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:53.706579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.706960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:53.707247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.707532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.707627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:53.707810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.707902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:53.708146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:53.708639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] RBAC: Adding query filter params (group.id=2417a4a63c334b8d983365c0b942d028, role.id=8182238074534b76a65d54c2edbeffad, scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:53.708950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.709032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] RBAC: Adding query filter params (group.id=3d214cbbea2a4a31bab7fef576a7f690, role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c113e9e1592248a9b12fd4d2b385aa5a) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:53.709281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:53.711021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33231c4e-1e87-4fd9-aa74-a9d4ad37b09a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.711280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2d030-0a90-43df-a8f1-420d6b63177d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:53.716782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 230/454] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/role_assignments?group.id=2417a4a63c334b8d983365c0b942d028&scope.project.id=9a2df333a2bd4f919ac91e69ef586e3c&role.id=8182238074534b76a65d54c2edbeffad => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:53.717788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 225/455] 10.222.0.39 () {56 vars in 1376 bytes} [Wed Jul 13 20:51:53 2022] GET /identity/v3/role_assignments?group.id=3d214cbbea2a4a31bab7fef576a7f690&scope.project.id=c113e9e1592248a9b12fd4d2b385aa5a&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:56.680574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90407dd-c6a3-4db4-bc88-3c3c256ac29d None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.680574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90407dd-c6a3-4db4-bc88-3c3c256ac29d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.680574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90407dd-c6a3-4db4-bc88-3c3c256ac29d None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.681689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 231/456] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:56.689063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31dd12-9f9b-4881-93a6-50987d4acaf7 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.689973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31dd12-9f9b-4881-93a6-50987d4acaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.690511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31dd12-9f9b-4881-93a6-50987d4acaf7 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.692318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 226/457] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:56.693975 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.694457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.695527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.697266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c932a1-346f-4af2-928a-581ccea9c70e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.697752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c932a1-346f-4af2-928a-581ccea9c70e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.698225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c932a1-346f-4af2-928a-581ccea9c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.726445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:56.730709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:56.731791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-39c932a1-346f-4af2-928a-581ccea9c70e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:56.733132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-39c932a1-346f-4af2-928a-581ccea9c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:56.800783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:56.802925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 232/458] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:56.827933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:56.871408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 227/459] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:56.872969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc36354-b235-410b-8fc4-5d0b0e38a317 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.878643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc36354-b235-410b-8fc4-5d0b0e38a317 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.879165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc36354-b235-410b-8fc4-5d0b0e38a317 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.880668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 228/460] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:56.891898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.892432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.892789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:56.916581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:56.917868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:56.948868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636182cc-da32-4c40-9dc5-ce40acee66d2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:56.990602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:56.991705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:56.992135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:56.992681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.005486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.007788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.009074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 229/461] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:51:57.009922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.020738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:51:57.040512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:57.041424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:57.042303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:51:57.042303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:51:57.042303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:57.042303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:57.044627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 233/462] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:51:57.050409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.106405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.142351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad220d7-fc23-4ce8-aaee-35115ffe12e3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.170931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c40fc-865e-42c0-bbe0-252d9771af29 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.179617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.180749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.181231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.181690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.192725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.194584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.210786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.211383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.211668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.211943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.212672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:57.212978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.214330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:51:57.218726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:57.219532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:57.220354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:51:57.220354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:51:57.220354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:57.220354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:57.222400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 230/463] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:57.223379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.234739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 234/464] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:57.235859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.299119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.349026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26a0b9-cb59-4b17-ad73-ea69105d7d25 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.365262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604a3ff-3a27-4599-bc29-abacfb9d3658 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.377222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.387776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.388828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.389174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.389635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.400243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.401913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.402416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.403106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.403388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.403661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.404385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:57.404688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.406035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.418201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 235/465] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:57.419009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-f8b12e63-fa0a-4614-9789-7fd2a3dc59ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:51:57.422192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:57.423064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:57.423869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:51:57.423869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:51:57.423869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:57.423869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:57.424070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 231/466] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:57.425204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d254d-5c59-4262-bb4b-053330c17a8f None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.425490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d254d-5c59-4262-bb4b-053330c17a8f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.425766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d254d-5c59-4262-bb4b-053330c17a8f None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.427064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 232/467] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:57.429740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d254d-5c59-4262-bb4b-053330c17a8f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.531431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce237c-d235-4bfc-8db1-2cb45d18b5ad None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.536725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d254d-5c59-4262-bb4b-053330c17a8f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.545138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.568332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.568997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.569281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.569585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.570377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.571856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.573569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3fff0-c47b-4a52-99fc-9f1c1179d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.574239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3fff0-c47b-4a52-99fc-9f1c1179d6d3 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.574524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3fff0-c47b-4a52-99fc-9f1c1179d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.574829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3fff0-c47b-4a52-99fc-9f1c1179d6d3 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.575529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3fff0-c47b-4a52-99fc-9f1c1179d6d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.576967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3fff0-c47b-4a52-99fc-9f1c1179d6d3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.594755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 236/468] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:57 2022] POST /identity/v3/users => generated 248 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:57.595588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.597172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 233/469] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:57 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:57.598736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.599256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.599756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.630587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:57.634122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:57.667080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.780678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 234/470] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:57.802163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda212-c339-4d58-af2a-182c0c02d995 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.817272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.843796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:57.844462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:57.844746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:57.845024 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:57.845758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:57.846087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:57.847517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:57.861582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 237/471] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:57.867552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:57.949558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.961449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf88b6-0465-44ad-9ca1-7c838a57c5cd None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:57.969956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:57.986979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466326a-b8c7-4d45-9f85-56ec20653a92 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.009608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.010507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.010833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.011112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.022150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.024247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.025329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9eb08-7eac-4b23-8d17-6691da87cd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.026095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9eb08-7eac-4b23-8d17-6691da87cd61 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.026379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9eb08-7eac-4b23-8d17-6691da87cd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.026702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9eb08-7eac-4b23-8d17-6691da87cd61 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.027476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9eb08-7eac-4b23-8d17-6691da87cd61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.028888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9eb08-7eac-4b23-8d17-6691da87cd61 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-14f30b5c-d994-47ad-8c2d-5e1946e896e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:51:58.043754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:58.044648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:58.045629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:51:58.045629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:51:58.045629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:51:58.045629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:58.045629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 235/472] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:58.046597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a6506-7f15-42f2-9934-c71cbecfb9f3 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.046885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a6506-7f15-42f2-9934-c71cbecfb9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.047407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a6506-7f15-42f2-9934-c71cbecfb9f3 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.052895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 236/473] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:58.052895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a6506-7f15-42f2-9934-c71cbecfb9f3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:58.059122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 238/474] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:57 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:51:58.060461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.060760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.061102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.082801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:58.084277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:51:58.190759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 239/475] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:51:58.197381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a6506-7f15-42f2-9934-c71cbecfb9f3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.232533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:58.250753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.251442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.251723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.252190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.254038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:58.254351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.256757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.286068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 237/476] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:51:58.291692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:58.296786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:58.383952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:58.405080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6b8d0-0274-43d6-9f0a-64efbb9a5b3a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.424228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85afc8-e280-49a3-818e-0d343e579dde None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.445354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.446298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.446769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.447188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.448065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.450457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.469403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.470083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.470596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.475025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.476949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.479675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.480489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 238/477] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:51:58 2022] POST /identity/v3/users => generated 249 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:51:58.488509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.sql.core [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 20:51:58.488509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 20:51:58.488509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [parameters: {'id': 'e4851e3a85ca415988f07a1d72512309', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 20:51:58.488509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result.read() Jul 13 20:51:58.506507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return ref.to_dict() Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:51:58.508012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application yield resource Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise self.value Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application session.commit() Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 20:51:58.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.session.flush() Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application compat.raise_( Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 20:51:58.509785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:51:58.510678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result.read() Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:51:58.511510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [parameters: {'id': 'e4851e3a85ca415988f07a1d72512309', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:58.512267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 20:51:58.513123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:51:58.513950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 240/478] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Jul 13 20:51:58 2022] POST /identity/v3/roles => generated 144 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:58.518764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:58.587228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:58.648673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444803e-2dab-400b-a5ce-1cc929f89acb None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.693683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f07d-fb47-402b-b292-4cf90394172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.694589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f07d-fb47-402b-b292-4cf90394172d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.694899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f07d-fb47-402b-b292-4cf90394172d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.695173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f07d-fb47-402b-b292-4cf90394172d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.714490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f07d-fb47-402b-b292-4cf90394172d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-c3b9f07d-fb47-402b-b292-4cf90394172d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:51:58.722708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:51:58.724752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 20:51:58.726777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:51:58.730497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 239/479] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:51:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:51:58.730742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:51:58.856618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68f209-143d-4b4f-997c-6c86e024ed17 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:51:58.891331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:51:58.892007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:58.892290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:58.892568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:58.893330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:51:58.893634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:51:58.895036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe9df1-4443-4915-9a29-60924bd7c034 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:51:58.897317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:51:58.903893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 241/480] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:51:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:51:59.937598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e514c64-6aeb-4ead-bd5c-24770c5dfb73 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.938637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e514c64-6aeb-4ead-bd5c-24770c5dfb73 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.939129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e514c64-6aeb-4ead-bd5c-24770c5dfb73 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.940959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 240/481] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:51:59.946684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a337-d099-4136-ab7b-b1b52296c378 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:51:59.947508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a337-d099-4136-ab7b-b1b52296c378 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:51:59.947986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a337-d099-4136-ab7b-b1b52296c378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:51:59.965612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d6c0a337-d099-4136-ab7b-b1b52296c378 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:51:59.967108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d6c0a337-d099-4136-ab7b-b1b52296c378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.037081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 242/482] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:00.041208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbc630-75a0-4a2c-a879-6759f73ce1c7 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.041774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbc630-75a0-4a2c-a879-6759f73ce1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.042274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbc630-75a0-4a2c-a879-6759f73ce1c7 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.044125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 241/483] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:00.049299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.049587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.050078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.067660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:00.069200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.078127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbc630-75a0-4a2c-a879-6759f73ce1c7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.145376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 243/484] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:00.168810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.229724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.269803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1d4aa-4337-4588-8653-0b294fb1c34a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.275321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbc630-75a0-4a2c-a879-6759f73ce1c7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.309216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.310248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.310701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.311117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.312372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.323449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:00.331560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:00.332564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:00.333368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:00.335562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 244/485] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:00.336652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.337511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.337796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.338121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.341520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.358313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:00.374639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:00.375715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:00.376686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:00.379040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 242/486] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:00.384828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.457206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24852443-2f7f-46c4-aa79-25b3e41411b6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.478040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.497758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.498836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.499400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.499854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.500784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:00.501290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.503006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.512547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 245/487] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:00.523751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.579824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1acc2e-92b8-4cc7-aef1-137354ec27e8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.614253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.614962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.615244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.615514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.616918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:00.617113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.618727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.627058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 243/488] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:00.635459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.653697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb611ba-eacd-43fc-b007-c1098e7ca48c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.706504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.711756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.712988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.713657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.714118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.729525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.731419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-40daaeac-8818-441a-8ba1-0aeb6335f081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:00.755173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:00.756011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:00.756869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:00.756869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:00.756869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:00.756869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:00.759939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 246/489] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:00.762019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2024ae-936f-4244-b06f-7c55f71949dc None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.762785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2024ae-936f-4244-b06f-7c55f71949dc None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.763243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2024ae-936f-4244-b06f-7c55f71949dc None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.765588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 247/490] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:00.766574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2024ae-936f-4244-b06f-7c55f71949dc None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.776354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a449b94-bbac-4c3f-9e57-17d05c3bf502 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.806568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:00.822022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.823359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.823718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.824401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.837993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.840313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a7cb2bda-abdd-451a-a3d4-5fa046843e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:00.862993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:00.863848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:00.864670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:00.864670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:00.864670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:00.864670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:00.864670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 244/491] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:00.866646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.866646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.866646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.886975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:00.888168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:00.892406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2024ae-936f-4244-b06f-7c55f71949dc None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:00.932579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:00.933298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:00.933584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:00.934004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:00.934818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:00.935127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:00.936716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:00.950067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 248/492] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:00.951014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:00.966932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 245/493] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:00.967816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.011051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.062875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.075948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d0127-9145-4d77-b59b-6c2fd7f2ffbf None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.091903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334e7a2-f76b-4e80-b8ca-773578c61e89 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.115700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.116595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.117040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.117512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.118499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:01.118980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.120607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.128890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.129897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.130328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.130808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] PATH_INFO: `/v3/users/glance` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.134842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 249/494] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:01.135836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.143070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.144746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:01.161904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:01.163923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:52:01.165058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:01.170939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 246/495] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:01.171935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.247337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b72a8-83ec-4599-82ce-dc3afb2c50d2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.260319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.283047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc80e-6c33-4c9a-9447-aefcc649bafe None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.297217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.298115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.298407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.298714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.311458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:01.320410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:01.321476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:01.322476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:01.323788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.324725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.325277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.325687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.326854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 250/496] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:01.327699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.334165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.336421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.344141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:01.352245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:01.353281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:01.354285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:01.356123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 247/497] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:01.357108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.445529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0a5a-1139-4435-8c6b-f729ce953edc None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.462461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94503d-f19e-4b7a-820d-a14566012868 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.485160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.485811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.486096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.486371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.487350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:01.487664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.489205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8929de3-e264-4cdd-a92b-782a6fcfe87a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.499407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.500280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.500713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.501133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.502085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:01.502643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.504260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.514177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 248/498] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:01.515221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.525110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.533081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 251/499] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:01.534407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f10b66-a07f-42ec-9f78-5f1777dda10c None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.534730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f10b66-a07f-42ec-9f78-5f1777dda10c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.535007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f10b66-a07f-42ec-9f78-5f1777dda10c None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.536384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 252/500] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:01.539963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f10b66-a07f-42ec-9f78-5f1777dda10c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.611654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.621598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1889c-000e-4196-b35e-b9acc9ec40ea None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.643765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f10b66-a07f-42ec-9f78-5f1777dda10c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.655452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.656144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.656428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.656704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.657536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:01.657844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.659287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.676275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.677051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.677330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.677601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.687946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.689390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.695552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 249/501] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:01.696420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ae27927b-7aa7-484f-8e1d-361f5cbd277a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:01.712280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:01.713268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:01.714080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:01.714080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:01.714080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:01.714080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:01.714080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 253/502] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:01.714735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222bd0-5a15-4980-8cd7-5ec177edb735 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.715022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222bd0-5a15-4980-8cd7-5ec177edb735 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.715295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2222bd0-5a15-4980-8cd7-5ec177edb735 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.716546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 254/503] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:01.718735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.718735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.718735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.735840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:01.737031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:01.762093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.799262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7e86f-6400-4f8b-8d9d-a338a7b9dc40 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.803037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 255/504] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:01.804355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:01.847506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.848147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.848429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.848704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.855501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:01.907135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc60a-f98e-4325-989a-b26f896ac481 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:01.917706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:01.919211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:01.941412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d1638-f8d4-4e0c-98eb-690951dad820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:01.942064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1638-f8d4-4e0c-98eb-690951dad820 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:01.942351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1638-f8d4-4e0c-98eb-690951dad820 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:01.942655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1638-f8d4-4e0c-98eb-690951dad820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:01.978699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.009618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d1638-f8d4-4e0c-98eb-690951dad820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.011110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d1638-f8d4-4e0c-98eb-690951dad820 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.016420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 250/505] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:02.017247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.060935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.108125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 256/506] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:02.109485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.109781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.110069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.115507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcd0a8-c733-4f07-9166-35ac50d7cf1d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.126179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:02.127393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:02.148482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.149136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.149422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.149694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.150421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:02.150760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.152119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.163868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 251/507] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:02.164632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.188878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.202676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 257/508] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:02.202676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.262119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa01ba3-c8fd-4ef6-95f3-009bca970bfc None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.282173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.299288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.300083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.300369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.300649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.301240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b3065-8cee-4b19-98d1-76db04bce4f3 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.311065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.312489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:02.329246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:02.330101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:02.331016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:02.331016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:02.331016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:02.331016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:02.331016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 252/509] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:02.331530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.338774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.339572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.339854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.340124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.350295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.352343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:02.369476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:02.370325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:02.371189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:02.371189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:02.371189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:02.371189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:02.371189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 258/510] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:02.371466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.410766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.433910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af4d1-0995-490a-99e4-0dcde137177f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.466134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.466939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.467225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.467497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.472679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b265629-dad6-457b-b067-cb35c705a380 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.478223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.479695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:02.496108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:02.497134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:02.498019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:02.498019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:02.498019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:02.498019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:02.498019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 253/511] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:02.498302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.509900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.510722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.511005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.511285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.513186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.520034 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.521473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:02.537037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:02.537920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:02.539664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 259/512] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:02.540220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.600236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f3def-8be2-4f9e-8ef5-9c1bf9fbb62d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.625881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.634779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce7556-43a0-4d0c-8aa2-09852d9c35c3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.635176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.635966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.636256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.636530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.646426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:02.654948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:02.655993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:02.656843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 254/513] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:02.657479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.671073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.671734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.672018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.672297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.673022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:02.673333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.674721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.686169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 260/514] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:02.686969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.741753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.762754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a660c37-1d84-47ac-a652-2fe722c7d1ef None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.784820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cd261-2e7c-423a-9dc3-aad484211e7b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.795375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.796029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.796311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.796588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.797335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:02.797640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.799040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.810506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 255/515] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:02.811332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.817866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.818607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.818921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.819206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.820460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:02.820771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.822129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.833443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 261/516] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:02.834218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.864733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.914748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8874e31-ace4-480b-8c15-1a8ac609b6c6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.932352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058476ed-7e4e-4005-ab76-8fcd4e1d47dc None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:02.945863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:02.947542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.948207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.948491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.948775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.949605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:02.949917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.951359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.966747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:02.967409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:02.967696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:02.967978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:02.968590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 256/517] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:02.968754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:02.969035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:02.969359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:02.970407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:02.978644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 262/518] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:02.979365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.072336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da581d3-6948-4880-8991-bb35c259b789 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.079410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.079917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b11bc7-f0b0-4d2b-92ea-c449d0298b7a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.106480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.107171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.107461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.107736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.108444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.109869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.111545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.112324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.112604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.112872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.119972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.121381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.129103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 257/519] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:52:02 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:03.129719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:03.137556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:03.138481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:03.139380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 263/520] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:03.139861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.187550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.245966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e61a1-71da-4ea9-80f5-68481f89440c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.263242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9889-dc0a-4101-b307-d2c90f9ff56c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.287422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.288270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.289236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.289524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.297467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.298971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.303340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.304487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.304770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.305040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:03.316143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:03.316980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:03.317800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:03.318367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 258/521] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:03.319091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.344515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.383539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.385244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.423372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.434149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd3f6d-c500-4f2d-a054-e636c5118f80 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.470259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.471394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.471836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.472119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.484171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.485628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.494585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 264/522] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:03.495534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:03.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:03.509382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:03.510211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:03.510211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:03.510211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:03.510211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:03.510211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 259/523] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:03.510588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.588128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.609315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e51492-12f2-4cce-a1d8-812ed02189fe None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.617372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2aa40-a661-4e23-a1e4-30561a707d78 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.643606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.644247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.644533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.644815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.645652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:03.645959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.647441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.651676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.656091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.656752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.657973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.659376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:03.659691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.661124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.671532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 265/524] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:03.671532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.681344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 260/525] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:03.682030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.782889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1748c41-780d-4318-ac58-762cb2245f06 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.783376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad46611-13a3-4b38-bf8a-676ecfbc19a4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.818521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.819376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.819659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.819930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.820489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.821152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.821436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.821719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.822452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:03.822783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.824175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.825759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.831821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.833270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.836727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 261/526] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:03.837670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:03.853422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:03.854426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:03.855447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:03.855447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:03.855447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:03.855447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:03.856987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 266/527] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:03.857917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:03.918433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:03.952594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdf83d-8e98-4745-a2c5-21379f4d9a9b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.972652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51f5c7-0164-4f59-9f31-af1d4c611883 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:03.986959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:03.987721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:03.988101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:03.988490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:03.989415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a6f4dc9555784337b2fd930caea78fde) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:03.989824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:03.991476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:03.997150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 262/528] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/role_assignments?user.id=a6f4dc9555784337b2fd930caea78fde&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:03.998008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.028070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.028759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.029032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.029315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.030291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a2048d28ca3b42d8b8d82f4754e44433) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:04.030610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.032184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.040185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 267/529] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/role_assignments?user.id=a2048d28ca3b42d8b8d82f4754e44433&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:04.041158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.108800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:04.173460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a045b-ca48-4f4d-be1e-b98e05c70c34 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.191878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:04.223346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12652717-6deb-425c-bd37-2f11d5cb1198 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.224354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.225047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.225332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.225654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.226437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:04.228339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.229138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.242730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 268/530] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:04.248117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.286665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.286665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.286665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.286665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.302893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.304758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:04.325802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:04.326765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:04.327851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 263/531] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:04.335472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.393085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5d69c-1743-4ce6-a95d-081cb1ca61c2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.436762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.437494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.437780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.438060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.460398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.461871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.485304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6dcaf-bb2f-4ff8-afcf-17fc35e03c31 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:04.495013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:04.496994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:04.499113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 269/532] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:04.502316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.503227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:04.535543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.536220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.536503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.536780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.537633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:04.537979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.539695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.561194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 264/533] 10.222.0.39 () {56 vars in 1139 bytes} [Wed Jul 13 20:52:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:04.570980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:04.572424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.662916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf364ad-3a8f-4b10-86b4-7d0713e3d801 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.737467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822dc0a-b687-4e5c-a9cd-0b0089336a9f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.757339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.758029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.758321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.758601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.759430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:04.759738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.761362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.783919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.784940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.785434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.785899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.794122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 270/534] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:04.800199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:04.902986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:04.955013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:04.959187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9541c68-439c-4db9-bfa2-a2d4bb8c12b6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:04.959475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:04.961001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:04.997086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:04.997798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:04.998081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:04.998441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:04.999358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=9a58cb03e518413ab12cc84d793c4736) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:04.999670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:05.001444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:05.007847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 271/535] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:04 2022] GET /identity/v3/role_assignments?user.id=9a58cb03e518413ab12cc84d793c4736&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:05.103576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 265/536] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:05.111801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.242082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25000c-77b2-40b6-bab8-ed4c36af6b3d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:05.289994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:05.290923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.291212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.291487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.307246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:05.308725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-586f2346-f76d-4c05-b95d-ca10513baca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:05.336181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:05.337076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:05.338583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:05.338583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:05.338583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:05.338583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:05.338583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 272/537] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:05.342891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.354309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce44280-2e42-4bc9-a97c-1305eb2c7774 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.354610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce44280-2e42-4bc9-a97c-1305eb2c7774 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.354909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce44280-2e42-4bc9-a97c-1305eb2c7774 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.359976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 273/538] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:05.371226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.371517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.371791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.396591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:05.398478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:05.402930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:05.430793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:05.454225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab67739-a372-4a4e-8aac-ad6c537f1cf9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:05.495343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:05.496390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.496873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.497375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.498297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:05.502642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:05.505482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:05.509166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 274/539] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:05.519021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 266/540] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:05.525053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.536904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.697652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aeafe-03c8-40fa-8f32-45571092ea2f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:05.705362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e38211-80ad-48c8-bc0d-0dbec2d1a4e4 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:05.745716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:05.746517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.746825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.747101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.751432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:05.763180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:05.770951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:05.771832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:05.773647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:05.774241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:05.774241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:05.774241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:05.774442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 267/541] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:05.774507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:05.781046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.795831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:05.798460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:05.803603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:05.826502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:05.827401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:05.828216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:05.834153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 275/542] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:05.837646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:05.954651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b156942-d826-4467-b807-7c35006b7c91 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.012482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9c0e1-1e46-45a4-a0f6-f8ab0ddbe6c4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.024090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.024779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.025063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.025341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.026118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:06.026446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:06.027824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:06.045994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 268/543] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:06.052318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.057298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.058303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.058810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.059305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.060345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:06.060845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:06.062473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:06.080919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 276/544] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:06.082491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.166297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:06.183715 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:06.219206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c928d1-c2a2-45af-8fc2-e0ecc819bdf6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.239625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19622c9-b3b0-4d1f-a21a-7ee0d20f2bc3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.279635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.280110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.280355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.284852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.284852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=df6ccbd929724922bdbb1150d543bacd) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:06.284852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:06.284852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:06.290936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.291749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.292032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.292308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.302941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 277/545] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/role_assignments?user.id=df6ccbd929724922bdbb1150d543bacd&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:06.320553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:06.322045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-c2b7d17f-52bc-425f-b8ab-46db7cb074b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:06.359253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:06.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:06.361891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:06.361891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:06.361891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:06.361891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:06.361891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 269/546] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:06.366741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.440722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cbf28-b53d-4559-9013-a0a8fc3b33c4 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.441175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cbf28-b53d-4559-9013-a0a8fc3b33c4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.441556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cbf28-b53d-4559-9013-a0a8fc3b33c4 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 270/547] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:06.452679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fced3c-4c70-4705-a9fd-7fcebae4471d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.452960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fced3c-4c70-4705-a9fd-7fcebae4471d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.453244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fced3c-4c70-4705-a9fd-7fcebae4471d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.479294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-69fced3c-4c70-4705-a9fd-7fcebae4471d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:06.480498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-69fced3c-4c70-4705-a9fd-7fcebae4471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:06.521387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e5f95-9029-4bdd-9edd-781a4cd48a7a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.554382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:06.556599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 271/548] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:06.558197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47302f-a46b-4296-b3d6-a11d4b1fbfaf None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.558576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47302f-a46b-4296-b3d6-a11d4b1fbfaf None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.559038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47302f-a46b-4296-b3d6-a11d4b1fbfaf None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.564452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 272/549] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:06.570470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.570470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.570470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.580980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.581881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.582414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.582880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.583757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:06.584313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:06.585232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:06.586053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:06.586665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:06.602592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 278/550] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:06.607732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.631835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:06.676417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 273/551] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:52:06.677685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.737910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08078f48-c929-4284-b0c8-09feab06cb53 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.776137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.777079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.777450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.777897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.789430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:06.797187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:06.798816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:06.799983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:06.801443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 279/552] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:06.802421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.813424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25691dc7-8c7a-4523-b540-d74cce8116d7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.847593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:06.864743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.866086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.866954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.867521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.878776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:06.880431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:06.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:06.901819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:06.903051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:06.908860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 274/553] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:06.908860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:06.924386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b887e-344d-453d-9746-cda6172487e3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:06.940586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:06.976997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:06.977797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:06.978077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:06.978349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:06.989357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:06.997339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:06.998284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:06.999141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:07.002712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 280/554] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:07.003683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.032281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224bc1e-c844-4e21-9913-88f3d39191d9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.086568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.087221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.087507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.087779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.088522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:07.088825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.090176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.098845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 275/555] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:07.099719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.138665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713866f-e2bb-48a4-84c1-7bf2223b302d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.141052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.188927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.189965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.190450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.191037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.191844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:07.192151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.193577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.199235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.211796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc7a36-e20d-4216-896c-2a16d6439c71 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.217846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 281/556] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:07.217846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.265415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.266940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.267697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.268236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.269871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:07.270397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.272802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.291041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 276/557] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:07.291041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.332146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf501a5-8f7d-4404-8a45-96f57c9f4720 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.368894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.370077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.370536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.371012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.382303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.384066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:07.402711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:07.404112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:07.405018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:07.405018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:07.405018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:07.405018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:07.407391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 282/558] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:07.408582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.420618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3784c25-9ceb-4ec1-b70a-d9504ba98faa None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.429994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.458657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea296b-796a-4ce7-a59c-cbe2c3fbe74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.459733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea296b-796a-4ce7-a59c-cbe2c3fbe74b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.460713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea296b-796a-4ce7-a59c-cbe2c3fbe74b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.461209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea296b-796a-4ce7-a59c-cbe2c3fbe74b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.486336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.531495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9cdb7-3f1d-45b6-ae72-6d298c875cdf None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.533185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea296b-796a-4ce7-a59c-cbe2c3fbe74b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.534664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea296b-796a-4ce7-a59c-cbe2c3fbe74b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.567044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.567941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.568221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.568494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.579558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.580994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:07.597797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:07.598757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:07.599638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:07.599638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:07.599638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:07.599638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:07.599638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 283/559] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:07.599904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.639996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 277/560] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:07.641598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79253e52-4b4a-46a7-bfb7-7c66485ca44f None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.641960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79253e52-4b4a-46a7-bfb7-7c66485ca44f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.642313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79253e52-4b4a-46a7-bfb7-7c66485ca44f None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.643820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 278/561] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:07.648325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79253e52-4b4a-46a7-bfb7-7c66485ca44f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.675406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.705331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.718827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0157ea-29f0-4f57-9d4d-b549ae9e3281 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.753493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79253e52-4b4a-46a7-bfb7-7c66485ca44f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.754940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.755600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.755886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.756159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.757430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:07.757733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.759130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.775420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 284/562] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:07.775420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.795785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.796455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.796738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.797019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.797750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:07.798055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.799455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08f8a8-85d4-497e-a83a-40b5907ada7f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.811112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 279/563] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:07.812306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.812654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.812936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.830458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:07.831668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:07.886213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2e1b5-928d-4224-941d-4e616e883e82 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:07.908165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 280/564] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:07.909095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:07.917472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.931933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:07.932879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:07.933270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:07.933651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:07.948038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:07.949546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:07.965366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:07.969025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:07.969880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:07.970812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:07.970812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:07.970812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:07.970812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:07.970812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 285/565] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:07.971107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.026435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a505af-2b0a-4632-aff1-abed18e82281 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.062582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.063443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.063727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.063997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.071594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.073051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.080669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bbf5-cd20-4481-ae32-a03e5edcbbed None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:08.087115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.088106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 281/566] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.089087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.120489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.121547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.121993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.122270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.130002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.131497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:08.145492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.146414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 286/567] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:52:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.147435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.190913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.202319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3539501-5b88-4f1a-9e0d-0f0ed7d12285 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.204667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.246402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.247564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.248048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.248928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.260655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c236-fbe8-414d-aa22-c3d12e3edb79 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.261643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:08.269576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.270534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:08.271473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:08.276253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 282/568] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.277130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.301028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.301691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.301972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.302245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.303013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:08.303307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.304794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.321208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 287/569] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:08.322114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.382667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb610c-7aea-4661-96f8-00b5a41b4173 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.416254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.417156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.417829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.418129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.418387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.419251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:08.419563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.420977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.423034 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35cfa1-fc32-4f2a-b4d4-cd923adb23c8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.438606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 283/570] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:08.439862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.456365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.456973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.457275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.457535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.458377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:08.458711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.460659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.464093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.478277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 288/571] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:08.479103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.541198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248df328-74b4-46bd-ac29-cd6cf87139ad None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.572891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533b3da-07f7-4d4e-8143-e9f14a01c92b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.574373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.575068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.575354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.575634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.576387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:08.576698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.578063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6dbd3-9006-485d-b0ee-ede95b83a572 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.586437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 284/572] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:08.587669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c388197-18da-43f5-a211-dbfe5273f738 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.587958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c388197-18da-43f5-a211-dbfe5273f738 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.588240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c388197-18da-43f5-a211-dbfe5273f738 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.589618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 285/573] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:08.590154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c388197-18da-43f5-a211-dbfe5273f738 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.608311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.609108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.609392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.609668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.616290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.617721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:08.631203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.632208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 289/574] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.650066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.676579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.694682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c388197-18da-43f5-a211-dbfe5273f738 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.728403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.729165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.729452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.729733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.734255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420b740-fb17-40d2-956e-5747d62370e4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.740406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.741959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:08.759141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.760149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:08.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:08.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:08.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:08.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 286/575] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.771680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.773124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.773403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.773678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.784139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.785598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-02ea0a37-a68f-4363-813c-bc63eacace98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:08.802432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.803462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.804454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:08.804454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:08.804454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:08.804454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:08.804454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 290/576] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.804861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.805082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.805357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.821413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:08.822593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:08.860149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03dee6-4213-451d-a506-80ac37753cf9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:08.880316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.888993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 291/577] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:08.889795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.899764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:08.900561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:08.900851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:08.901132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:08.904324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:08.914245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:08.915721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:08.932292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:08.933305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:08.934404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:08.934404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:08.934404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:08.934404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:08.934404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 287/578] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:08.934761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:08.992132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74bb40-c032-4ba2-a7cc-d1f274423f6c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.025920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.026578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.026892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.027173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.027967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.028281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.029700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.031648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980df00-b0ec-430d-b8e8-9cbcb273c880 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.041154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 292/579] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.041950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.068798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.070070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.070351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.070642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.071369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.071675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.073023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.084337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 288/580] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.085131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.099162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.133887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.142521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad93c-027d-4570-a835-a2f87ec87464 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.174483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.175180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.175471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.175751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.176477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.176842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.178233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.184026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43c74f-ed3a-4644-b469-7550f3c96682 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.190120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 293/581] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.190929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.217466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.218251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.218534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.218844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.228532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:09.236054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:09.236949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:09.238039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 289/582] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:09.238943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.298349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e39e2-13ca-47aa-8427-ae989161359c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.316198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.334640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.335326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.335605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.336594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.336894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.338289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.340441 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb166475-8bce-4973-9a69-66962107dcd9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.349008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.349199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 294/583] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.349975 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.376963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.377607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.377883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.378155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.379007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=292fed9f5de945c883cc1ffeb91da4cb) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.379312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.380729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.386541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 290/584] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/role_assignments?user.id=292fed9f5de945c883cc1ffeb91da4cb&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.387323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.456564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87c8da-91da-4c26-b4ae-206d57c82faf None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.496810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.497766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.498054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.498328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.506042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.507587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.519514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f55593-4284-456d-907f-98660a0e32d5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:09.522790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:09.523722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:09.525059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 295/585] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:09.525793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.563129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.563922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.564209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.564488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.566629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.572876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.574351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:09.588909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:09.589834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 291/586] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:09.591291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.625250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.644140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4978b-9989-483a-9495-5c87083eb0be None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.680790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.681472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.681737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.682013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.682794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.683107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.684469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.693052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 296/587] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.693857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.703901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a7436-0afb-4ac4-8644-6d0fd554df7f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.745680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.746570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.746940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.747278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.754481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.756015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:09.770372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:09.771535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:09.773202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:09.778946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 292/588] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:09.778946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.800059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.804761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf05e76-de38-43e8-9818-32cb502cfef6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.843091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.843763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.844052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.844333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.845125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.845443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.846893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.859878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 297/589] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.860717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.880992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:09.890779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3f4ba-943b-4206-90e2-768c2177e970 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:09.928868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:09.929627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:09.929972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:09.930308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:09.931243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:09.931705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:09.933095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:09.949558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 293/590] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:09.949558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:09.974578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79ed34-b676-4aae-9666-1a32c50d14fe None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.010033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.010896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.011186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.011475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.023038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.024492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:10.043244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:10.044293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:10.050095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:10.050095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:10.050095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:10.050095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:10.050095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 298/591] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:10.050095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.059575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066ebc-fd03-4d1d-9a9e-d68ac9bd1b91 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.059984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:10.097004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.097672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.097949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.098223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.099214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:10.099519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.101658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.125655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 294/592] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Jul 13 20:52:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:10.125655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.138796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:10.156151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2c281-7688-4fdc-a33c-8f05c985434c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.192099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.192979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.193260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.193529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a6f4dc9555784337b2fd930caea78fde/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.211944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=a6f4dc9555784337b2fd930caea78fde, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.213368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.226477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 299/593] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:10 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a6f4dc9555784337b2fd930caea78fde/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:10.229925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad808925-d28c-48a2-9629-9e8cf0f19ebd None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.231950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.277995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.279191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.279624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.280047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a2048d28ca3b42d8b8d82f4754e44433/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.298598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=a2048d28ca3b42d8b8d82f4754e44433, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.300375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.319841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 295/594] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:10 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a2048d28ca3b42d8b8d82f4754e44433/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:10.321204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.331131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:10.409803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bada2a-21ab-4e2b-ac21-f1a469c3b952 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.430228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:10.450307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.451278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.451704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.452151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.453041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:10.453490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.455064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.469451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 300/595] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:10.475829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.481184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60da3e-ce28-4fd4-88cf-8ae190790530 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.527630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.528431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.528713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.528990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.540085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.541543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:10.562751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:10.563646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:10.564523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:10.564523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:10.564523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:10.564523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:10.564523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 296/596] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:10.570036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.606339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330c1ac-8621-48d9-ae7d-32f94799eaa3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.628074 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:10.644544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.645354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.645634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.645911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.654211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.655772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.674086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeb072-69a8-4d25-880d-ee81b92fc643 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:10.675632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:10.676704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:52:10.677836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:10.680510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 301/597] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:10.689069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.689222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:10.713656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.714325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.714608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.714900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.716268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:10.716574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.717940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.739685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 297/598] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:10.745972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.875837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617cfc4-3bf4-4f88-bc12-1b72dd044b26 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.885481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4b8b-0d2f-48d1-82f3-83e35c049d52 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:10.911204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.911998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.912277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.912549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.927956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.930005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.946940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:10.947896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:10.948286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:10.948620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:10.949608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:10.950705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:10.951136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:10.951204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 298/599] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:10.951279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:10.954640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:10.954640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:10.959718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:10.979559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 302/600] 10.222.0.39 () {56 vars in 1139 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:10.986652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.018317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.064162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.146639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713bf3-8963-434a-891a-04504e4c7347 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.174719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ca079-72b2-497f-9c08-0a2b8870dc38 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.201025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.201696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.201978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.202256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.203082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:11.203398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.204944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.229430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 299/601] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:11.232990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.241515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.242320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.242604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.242935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.265707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.267229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:11.300837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:11.301761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:11.302635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:11.302635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:11.302635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:11.302635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:11.306970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 303/602] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:11.308275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.429219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.446967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.451918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8f9bd-7106-4525-8e68-e9c8973e4ad2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.456659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a713d-d923-494f-b602-ab808aead5b5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.498434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fab6d-52af-4f5f-be2d-2ed7e814da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.499684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fab6d-52af-4f5f-be2d-2ed7e814da10 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.500164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fab6d-52af-4f5f-be2d-2ed7e814da10 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.500636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fab6d-52af-4f5f-be2d-2ed7e814da10 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/9a58cb03e518413ab12cc84d793c4736/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.500763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.501428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.501714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.501994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.502747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:11.503057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.504436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f174ab-d401-4a1a-997a-98c34cc2c118 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.519174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fab6d-52af-4f5f-be2d-2ed7e814da10 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=9a58cb03e518413ab12cc84d793c4736, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.519383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 304/603] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:11.520611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c4a19-ff2a-4420-96a5-367333658679 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.520898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c4a19-ff2a-4420-96a5-367333658679 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.521174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c4a19-ff2a-4420-96a5-367333658679 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.521855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fab6d-52af-4f5f-be2d-2ed7e814da10 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.526695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 305/604] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:11.527745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c4a19-ff2a-4420-96a5-367333658679 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.545214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 300/605] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:11 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/9a58cb03e518413ab12cc84d793c4736/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:11.547115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.547540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.547984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.567962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:11.570356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:11.656252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 301/606] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:11.681817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.761029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c4a19-ff2a-4420-96a5-367333658679 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.774960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.802917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.803959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.804358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.804726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.813155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:11.814788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:11.815861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343d192-39f1-45e0-9c09-3b2b09ad8d8a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:11.832586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:11.833714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:11.835493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:11.837281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 306/607] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:11.845500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.862563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:11.879476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:11.880404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:11.880786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:11.881173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:11.899431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:11.908065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:11.909205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:11.910301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:11.919307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 302/608] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:11.931634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:11.988521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0f33-75ad-4cc2-b9cb-b0c56e5a7385 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.032313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.033126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.034595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.036009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.036818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:12.037919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.039640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.069573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 307/609] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:12.075379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.111963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ecee-fe67-4be4-a8e4-eac50a8fd8a5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.156664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.179355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.180022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.180140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.180300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.180570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.181320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:12.181625 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.183022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.201973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 303/610] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:12.211635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.214058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25664774-8b15-4d02-b3bb-708b4d768653 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.254575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.255407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.255693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.255970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/df6ccbd929724922bdbb1150d543bacd/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.296716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=df6ccbd929724922bdbb1150d543bacd, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.298106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.334317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b7eae-6756-4c19-96ea-715371d48368 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.335085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 308/611] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:12 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/df6ccbd929724922bdbb1150d543bacd/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:12.372473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.373576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.374004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.374428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.388454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.390197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:12.409424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:12.410541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:12.411668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:12.411668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:12.411668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:12.411668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:12.414450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 304/612] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:12.419827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.554799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695ef55-4f1d-44c0-8ca9-e36a8c5f610b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.579452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.595458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.596187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.596542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.596825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.597570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:12.597967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.599758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.617704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 309/613] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:12.623039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.693631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:12.764230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c44fc-8fa3-4164-b6c7-d7ad07da0fbf None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:12.811553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:12.812263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.812544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.812820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.820803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:12.822326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-71b79f54-8360-4111-9645-3cd130392343 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:12.847847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:12.849016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:12.850786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 305/614] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:12.856966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:12.890783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fce02-6479-4502-b38e-1d968eb2111d None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.891072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fce02-6479-4502-b38e-1d968eb2111d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.891351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fce02-6479-4502-b38e-1d968eb2111d None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.892912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 306/615] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:12.902271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8314-a7a6-43b3-a529-5bc636fd56c3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:12.902554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8314-a7a6-43b3-a529-5bc636fd56c3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:12.902892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8314-a7a6-43b3-a529-5bc636fd56c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:12.928314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5c7c8314-a7a6-43b3-a529-5bc636fd56c3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:12.929496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5c7c8314-a7a6-43b3-a529-5bc636fd56c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:13.005862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4c359-c152-4d99-9f0f-b81b1ffb5a97 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.013222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 307/616] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:52:13.014890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463343-6ae8-4d1d-9547-da2daeacc368 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.015177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463343-6ae8-4d1d-9547-da2daeacc368 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.015448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463343-6ae8-4d1d-9547-da2daeacc368 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.016971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 308/617] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:13.024159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.024443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.024876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.052530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:13.052789 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.053551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.053751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:13.053855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.054155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.055041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:13.055429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.056950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.077104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 310/618] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Jul 13 20:52:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:13.082022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.096001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.129584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.151852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 309/619] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:13.152954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.195004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21defe0-9cf3-448c-8803-fbe4a031006b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.232615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.233560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.233949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.234329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.245784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:13.254081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:13.255272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:13.256905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:13.263583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 311/620] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:13.263583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.281197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30864a4e-cc23-4c07-99f5-02c7e41db86c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.323224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.324012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.324295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.324570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.342097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.343511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:13.360928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:13.361980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:13.362939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:13.362939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:13.362939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:13.362939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:13.362939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 310/621] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:13.363732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.383741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.401669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.407591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c3e0d-e5dc-406f-a54e-1efa92b3df11 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.453962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.455527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.455972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.456399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.472128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:13.479514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a770261-70cc-4b4f-83ff-a77d83f6c1b2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:13.483598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:13.484733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:13.485746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:13.488176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 312/622] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:13.489107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.519662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.520475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.521079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.521198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.522724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:13.523174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.524646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.533634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 311/623] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:13.538156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.619669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1e00b-102a-4163-aa43-4d0158804c19 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.643882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.665943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67974b94-e216-4ac6-b28d-69dd3e7ded51 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.668177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.668864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.669936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.670224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.671085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:13.671429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.672828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.688441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 313/624] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:13.689642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.709073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.709785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.710073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.710354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.711398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:13.711711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.712839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.714116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.725190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 312/625] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:13.726179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.827799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c344366-12af-4548-b353-d2837c8965bd None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.848367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c39221-c977-4acb-8f63-83f984ab6fce None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:13.864130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.865315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.866375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.866907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.878389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.880280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.890227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:13.891066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:13.891351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:13.891629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:13.898818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:13.900256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:13.901082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:13.901958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:13.902882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:13.902882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:13.902882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:13.902882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:13.905927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 314/626] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:13.907119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:13.916595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:13.917572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:13.919628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:13.923326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 313/627] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:13.924230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:13.928099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:13.992395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.027629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e6d2c-e7db-424e-a0aa-abf8c9206f20 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.064009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f14fd-7201-4845-b8c1-18b6a053dcd3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.068355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.069299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.069687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.070090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.081240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.082856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.099005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.099786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.100794 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:14.101055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:14.102052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:14.107772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:14.107772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:14.107772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:14.107772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:14.108044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.108129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 315/628] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:14.108208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:14.108288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.108378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.108378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a58a09-6cf8-477a-87a8-8fdb879bf52a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.123004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 314/629] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:14.124929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5758ca8-4e29-427f-8f6f-d1d5fc1c2841 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.125292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5758ca8-4e29-427f-8f6f-d1d5fc1c2841 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.125574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5758ca8-4e29-427f-8f6f-d1d5fc1c2841 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.127028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 315/630] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:14.127646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5758ca8-4e29-427f-8f6f-d1d5fc1c2841 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.193817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.215319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584301f-9e07-46f3-87c8-d544c8026064 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.238659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5758ca8-4e29-427f-8f6f-d1d5fc1c2841 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.239611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.253069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.253737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.254019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.254297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.255122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:14.255457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.256922 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.276258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 316/631] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:14.277152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.278915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.280091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.281466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.281762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.282534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:14.282876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.284255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108a4bc-96d5-4841-8849-2675a5922539 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.302303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 316/632] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:14.302303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.302303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.302303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.318070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:14.319324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:14.385298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43228bd-0f9a-4068-9758-928e9fe54ab2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.387848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 317/633] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:14.388655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.432605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.433402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.433713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.433992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.441390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.442874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.444925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:14.458598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:14.460332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 317/634] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:14.461507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.500130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daae42-5fed-4d10-8df6-9c940cfd91a8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.507681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.533630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.534507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.534812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.535093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/292fed9f5de945c883cc1ffeb91da4cb/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.549812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=292fed9f5de945c883cc1ffeb91da4cb, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.551305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.564021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 318/635] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:14 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/292fed9f5de945c883cc1ffeb91da4cb/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:14.564811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.565211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19daa1-e400-460e-91e7-d68ea1fcc4d3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.606686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.607668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.607958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.608235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.616734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.618253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:14.635005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:14.636485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 318/636] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:14.637431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.687714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71236653-a6c3-4816-aaeb-a610742824ff None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.689436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.733399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d3b5c-b2eb-49c0-bfd3-ae40399cd7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.734126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3b5c-b2eb-49c0-bfd3-ae40399cd7ee None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.734406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3b5c-b2eb-49c0-bfd3-ae40399cd7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.734703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3b5c-b2eb-49c0-bfd3-ae40399cd7ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.745091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d3b5c-b2eb-49c0-bfd3-ae40399cd7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:14.752137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd522f5-38da-40c2-b53d-e70f558e5909 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-673d3b5c-b2eb-49c0-bfd3-ae40399cd7ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:14.752737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:14.753824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:14.754825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 319/637] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:14.755683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc965-6d61-4f45-a1de-3b47d93cad76 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.755683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc965-6d61-4f45-a1de-3b47d93cad76 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.755831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffacc965-6d61-4f45-a1de-3b47d93cad76 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.760013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 320/638] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:14.760148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.760261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacc965-6d61-4f45-a1de-3b47d93cad76 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.800453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.801212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.801614 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.801889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.802752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:14.803140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.804722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.824740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 319/639] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:14.825564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:14.869748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffacc965-6d61-4f45-a1de-3b47d93cad76 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.905187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.905882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.906164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.906444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.907314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:14.907620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.909052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffe1df8-bcbc-4d01-b871-835074479593 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:14.927327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 321/640] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:14.928675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.928961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.929231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.940937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4b0d1-9f5f-4bb9-b31e-245c57d98b34 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:14.945595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:14.946801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:14.961832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:14.985451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:14.986052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:14.986336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:14.986645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:14.987379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:14.987683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:14.989437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.002785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 320/641] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:15.003757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.020186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 322/642] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:15.021303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.036908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.117985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a28b-a47f-4426-97c2-290332973f0e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.130433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee3fb-f599-4fab-8964-6372862d6cac None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.151586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.152236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.152518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.152793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.166862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.167719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.168180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.168722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.189541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.230132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.231616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.241815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.243235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.256388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.344933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 321/643] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:15.344933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.363570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 323/644] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:15.363570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.418376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.451321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183fa3b-da61-4df4-9a6b-8d9248d3106b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.472789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af75ec-71b1-4e40-93e9-5afd02050d67 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.476281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.488785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.489573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.489864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.490167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.500641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.502013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.508924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.509920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.510440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.510915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:15.520456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:15.521376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:15.523146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:15.523146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:15.523146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:15.523146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:15.523146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 322/645] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:15.523146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.524559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:15.532443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:15.533536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:15.534687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:15.535352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 324/646] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:15.536520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.636352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d616ac4-76a0-4af6-a8e1-e019fa3c9c0b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.639357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2392b-38d6-48cb-8590-7599da16d8ce None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.651359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.674834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.676029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.676467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.676810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.677109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.678216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.678504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.678813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.688409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.689646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.690076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.691116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:15.710289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:15.711232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:15.712124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:15.712124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:15.712124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:15.712124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:15.712375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:15.713151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:15.714051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:15.714051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:15.714051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:15.714051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:15.717215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 325/648] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:15.717327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 323/648] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:15.717420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.717493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.749067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.821081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aa517-a675-469f-8926-a7aca00e26ca None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.830240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f7d56-7d64-4444-b94a-83faabee1128 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.854589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.855279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.855562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.855836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.856554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:15.857085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.858213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.864521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:15.865174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:15.865456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:15.866261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:15.867027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:15.867330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:15.868671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:15.871635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 326/649] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:15.872617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.883843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 324/650] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:15.883843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:15.904131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.970290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:15.984439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beca72-423a-4f33-aaad-ad88a7287e4b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:15.987225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6be66-1d3d-49df-978a-268145138a9d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.020744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.021415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.021703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.021980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.022736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:16.023043 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.023654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.024310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.024411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.024585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.024860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.025578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:16.025887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.027283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.037265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 325/651] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:16.038004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.047480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 327/652] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:16.047480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.135193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.149575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c09f5c-b8f5-466f-81c7-7ebdad13d36f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.154168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e778c-809f-48f9-b9d3-72fca8381820 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.191023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.192248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.192480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.192974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.193086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.193323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.193625 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.193974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.203804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.204670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.205269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.206084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:16.219942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:16.220883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 326/653] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:16.227333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:16.228006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:16.228788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:16.232026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:16.232026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:16.232026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:16.232026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:16.232026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 328/654] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:16.232026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.241633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.335997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a4b00-85c8-4742-af08-a63610f44d8f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.344214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c23e9-acd5-469b-887b-4b2ceae53724 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.374440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.375268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.375553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.375833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.377935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.381154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.381980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.382346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.382538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.383503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.385494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.389453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.390910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:16.400227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:16.401072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:16.401931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 329/655] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:16.402849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:16.409131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:16.409986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 327/656] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:16.416949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.478546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.513448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77239fc4-3430-4a33-9ad6-204ca8253130 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.513753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43e62-9520-4e28-8beb-6db75cf3cf18 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.547590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.548257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.548544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.548822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.549553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:16.549860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.550465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.551183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.551290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.551464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.551747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.552703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:16.553092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.554650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.567062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 328/657] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:16.567062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.577190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 330/658] 10.222.0.39 () {56 vars in 1137 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:16.578145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.632630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.669846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb561-16a2-471f-96da-6bc02693bd80 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.679397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae99db-102a-4fd7-a617-175818f9b39c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.695905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.703445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.704104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.704388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.704664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.705441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:16.705750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.707228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.712851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.713507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.713792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.714071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.714886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:16.715197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.716610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabd048-3ed0-4a00-9e0b-47c33cb4fb3d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.723962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 329/659] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:16.723962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.733398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 331/660] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:16.733398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490086e9-39a4-4e8b-bd82-35a7325d7d6a None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.733398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490086e9-39a4-4e8b-bd82-35a7325d7d6a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.733398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490086e9-39a4-4e8b-bd82-35a7325d7d6a None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.733398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 332/661] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:16.733398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490086e9-39a4-4e8b-bd82-35a7325d7d6a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.820864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823377be-5de8-420c-9efc-62c63af22e72 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.830945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490086e9-39a4-4e8b-bd82-35a7325d7d6a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:16.843633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.854726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.855508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.855791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.856063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.863966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.865378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebef54-4b95-4c62-99d5-90492c53df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:16.865527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.866040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebef54-4b95-4c62-99d5-90492c53df0b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:16.866328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebef54-4b95-4c62-99d5-90492c53df0b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:16.866610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebef54-4b95-4c62-99d5-90492c53df0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:16.879157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:16.880190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 330/662] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:16.881324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:16.930728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:16.934804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebef54-4b95-4c62-99d5-90492c53df0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:16.936254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebef54-4b95-4c62-99d5-90492c53df0b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:16.980674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0479bd-e20c-4dae-bab9-4bb7cd9cfbe0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.012696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.013332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.013613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.013887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.014739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a6f4dc9555784337b2fd930caea78fde) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:17.015046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.016458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.022860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 331/663] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/role_assignments?user.id=a6f4dc9555784337b2fd930caea78fde&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:17.023451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.046706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 333/664] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:17.048753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.049443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.050351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.071236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:17.072483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:17.081079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.135914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610be6d-a7ca-4596-9619-f9cc28ebb00a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.150364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 334/665] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:17.150942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.151251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.189826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.190411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.190732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.191010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.191850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a2048d28ca3b42d8b8d82f4754e44433) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:17.192164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.194081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.206649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 332/666] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/role_assignments?user.id=a2048d28ca3b42d8b8d82f4754e44433&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:17.206649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.292685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a9eb0b-c649-4188-86c3-1c7ec3c1b85a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.338240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.338969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.339257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.339526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.340389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:17.340696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.340890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3002d-f377-4df7-b539-9b15e2ea47fa None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.342152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.358597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.366676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 335/667] 10.222.0.39 () {56 vars in 1139 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:17.367508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.390593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.391695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.391983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.392661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.408180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.409638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:17.429488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:17.430433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:17.432272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:17.432272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:17.432272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:17.432272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:17.432272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 333/668] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:17.432272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.474880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.510841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92edc-357f-45ff-96b1-8219b6a84f4f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.554730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.556070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.556890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.557648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.566526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766362f5-47e1-46b5-8188-94529ac9b6fc None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.571999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:17.581008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:17.581968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:17.582874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:17.583532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 336/669] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:17.589066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.617349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.618006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.618289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.618705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.655616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.706002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.707521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.710880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0087df-39cc-4689-b8c1-5afbb8f230ac None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.743451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.747311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.748060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.748392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.748668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.749505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:17.749888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.752012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.769938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 337/670] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:17.769938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.836253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 334/671] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:17.837110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.887257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012d21f-3b83-42c4-9de8-7943b2d9807f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.911334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:17.928525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:17.929189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:17.929473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:17.929747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:17.930529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:17.930830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:17.932190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:17.944947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 338/672] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:17.946211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:17.951578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1829b1-4172-4fa6-b061-cacaad399797 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:17.990246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.001907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.003237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.003953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.004811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.014565 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.016794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:18.036002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:18.037027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:18.038790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:18.044993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 335/673] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:18.045995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.062378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3069a-285d-49f6-ab95-953a81d5ddb1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.106545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.107891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.108504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.109107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.122360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.124401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:18.146501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:18.148098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:18.149193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:18.149193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:18.149193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:18.149193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:18.150522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84699bcb-f654-4e8a-b1e7-9aa39196a5a7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.151189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 339/674] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:18.152121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.192262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.192918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.194872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.196318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.196757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.209942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.212009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:18.233817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:18.234946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:18.236026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:18.236026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:18.236026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:18.236026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:18.237542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 336/675] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:18.238450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.286914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39badb-8263-42ba-a762-0101115045fa None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.297767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.340002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.341171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.341603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.342516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.343833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:18.344290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.346531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.361544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663c57-88c0-497d-a472-2a34aaf861d8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.370410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 340/676] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:18.371626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.401186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.401856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.402141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.402414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.403196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:18.403500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.405208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.423691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 337/677] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:18.424677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.486924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.495809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd84581-a059-410b-af93-262a98cb1ab1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.532186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.532778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.533058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.533332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.534343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:18.534673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.536218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.551213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.554758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 341/678] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:18.555804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.559476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a93de7-4193-4dd6-b363-b20c166212af None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.601320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.602353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.602889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.603731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.604764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=9a58cb03e518413ab12cc84d793c4736) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:18.605620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.607542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.622658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 338/679] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/role_assignments?user.id=9a58cb03e518413ab12cc84d793c4736&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:18.622658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.712153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656f79-35d3-4ea2-86bb-0d8c25a48348 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.727580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273514f7-3274-4839-8518-0bc1d117b9c1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.763049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.763862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.764153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.764402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.773751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.774218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.775302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.776101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.776588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.777162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.785482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.787859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:18.791908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:18.792842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:18.793751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 339/680] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:18.798936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:18.807911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:18.808849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:18.809797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:18.812239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 342/681] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:18.819429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:18.831556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.844935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:18.921430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c8cef-abe9-4ded-a71c-984fc89c5d1a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.966173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:18.966890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:18.967175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:18.967455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:18.968309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:18.968623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:18.970187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:18.993921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 340/682] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:18.996416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618ab06-2eef-42b8-a2b8-bf02621af2ed None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:18.999606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.045823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.046872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.047390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.047872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.048879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:19.049403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:19.051615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:19.071013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 343/683] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:19.077417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.107501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.128869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fd1f0-af54-41d2-b944-06c214fdabc0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.175463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.176211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.177435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.177980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.178502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.196054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b04a3-6889-4625-b6b4-ba27da4e71db None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.255567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:19.257916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:19.284016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.284638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.284930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.285207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.286114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=df6ccbd929724922bdbb1150d543bacd) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:19.286432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:19.292023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d7e9b-814c-45f2-8954-1419540ec243 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:19.305896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 344/684] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:19 2022] GET /identity/v3/role_assignments?user.id=df6ccbd929724922bdbb1150d543bacd&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:19.345775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de939a4-4c17-4492-896e-1c5db0100b72 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.346085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de939a4-4c17-4492-896e-1c5db0100b72 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.348963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de939a4-4c17-4492-896e-1c5db0100b72 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.348963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 345/685] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:19.358909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c35dd0-3df4-4ca4-a60f-30a90645389a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.359228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c35dd0-3df4-4ca4-a60f-30a90645389a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.359487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c35dd0-3df4-4ca4-a60f-30a90645389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.361262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.382484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-37c35dd0-3df4-4ca4-a60f-30a90645389a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:19.384310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-37c35dd0-3df4-4ca4-a60f-30a90645389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:19.415917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 341/686] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:19.421532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.498792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 346/687] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:19.500233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812659fc-9439-44b0-97b6-7b6dfb868fee None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.500574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812659fc-9439-44b0-97b6-7b6dfb868fee None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.500857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812659fc-9439-44b0-97b6-7b6dfb868fee None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.503125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 347/688] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:19.508535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.509052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.509547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.536121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:19.537668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:19.545877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e701-e132-4ad3-bff4-46d8542566e5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.589286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.590002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.590283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.590556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.602595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:19.604347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:19.616746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.623153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 348/689] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:52:19.623969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:19.631733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:19.632613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:19.633906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:19.633906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:19.633906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:19.633906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:19.633906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 342/690] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:19.638575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.677087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.753607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47352e-976c-49c9-82ad-5a9928d461f6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.769555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37e713-f1c8-4b58-96e3-67fb878c940b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.789822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.790638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.791009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.791357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.792167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:19.792559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:19.793998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:19.811165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 343/691] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:19.811165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.819305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038107-9346-429e-a736-21932c2cf51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.820277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038107-9346-429e-a736-21932c2cf51d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.820658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038107-9346-429e-a736-21932c2cf51d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.821030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038107-9346-429e-a736-21932c2cf51d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.831867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73038107-9346-429e-a736-21932c2cf51d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:19.833499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73038107-9346-429e-a736-21932c2cf51d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-73038107-9346-429e-a736-21932c2cf51d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:19.851741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:19.852834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:19.853912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:19.853912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:19.853912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:19.853912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:19.858758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 349/692] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:19.859756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038107-9346-429e-a736-21932c2cf51d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:19.919127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.926273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:19.936811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9c2cd-dbb7-4ce7-a177-4148e7abf28d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.978168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038107-9346-429e-a736-21932c2cf51d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:19.981656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:19.982647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:19.983076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:19.983516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:19.997771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:20.006915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:20.007996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:20.008960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:20.013817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 344/693] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:20.014723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.024232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.025303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.025760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.026570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.035580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.038388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:20.056205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:20.057078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:20.057938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:20.065239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 350/694] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:20.065239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.128009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a16f2d-e57f-4dc0-9f68-86bcb8c6fb63 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.164937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.165606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.165891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.166169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.167111 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:20.167303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.168691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.181764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 345/695] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:20.182604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.194791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728d0f6-d751-4458-9257-8ccf9a9502a6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.202943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:20.206988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:20.234646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.235911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.235911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.236339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.237198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:20.237818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.239285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.248070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 351/696] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:20.248793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.307650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d2ca2-3f0a-44c6-9e4b-e5d84666d741 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.351864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.353144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.353865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.354287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.355422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:20.355873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.357961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.364836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575dba5-3caf-4a30-9334-ac6aedf75477 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.377607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 346/697] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:20.377607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.414240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.415238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.415738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.416448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.418193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.419828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.448837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 352/698] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:52:20 2022] POST /identity/v3/users => generated 254 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:20.449977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.463790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:20.491870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:20.499754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bec9-3a28-41b4-8a13-f92ac98964e5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.581836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e683526-9295-4f49-bd29-e81cf6fb17a6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.599384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.600768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.601054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.601396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.620997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.622014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.622475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.623121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.624161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=292fed9f5de945c883cc1ffeb91da4cb) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:20.624649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.627005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.634356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 353/699] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/role_assignments?user.id=292fed9f5de945c883cc1ffeb91da4cb&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:20.639981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.641494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:20.676540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:20.677437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:20.678383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:20.678383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:20.678383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:20.678383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:20.683326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.687031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 347/700] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:20.800301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:20.810757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f8a92-9919-4d71-8dfc-efe87a778f7b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:20.850843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:20.851572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:20.851792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:20.852070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:20.852802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:20.853107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:20.854484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:20.867358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 354/701] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:20.872735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:20.986475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58efd68f-dfaa-47f8-8ee5-06cc002e1150 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:21.014344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:21.028722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:21.029531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.029816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.030093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.039255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:21.040762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:21.057070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:21.058003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:21.058941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 348/702] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:52:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:21.064190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:21.179573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7e90-eab7-4fc8-b14b-0b0497af86c7 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:21.216345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:21.217013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.217295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.217568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.218406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:21.218742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:21.220292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f316e-8677-4d04-9c08-51cb867d7b8f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:21.240806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 355/703] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:52:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:21.245913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:21.248382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536464f6-e52e-45cd-907e-b6eb3ef81542 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.248671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536464f6-e52e-45cd-907e-b6eb3ef81542 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.248945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536464f6-e52e-45cd-907e-b6eb3ef81542 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.250770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 356/704] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:21.260907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.261182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.261459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.300130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:21.301320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:21.309894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:21.381938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 357/705] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:21.407059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:21.520899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7acd25-cb9c-4182-9e90-b18b418f1897 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:21.523198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efc479-45a6-4055-ab6e-783e7ec3ee4c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:21.568321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:21.569537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.570047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.570521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.571346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:21.572268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7729a22-729c-42e2-957c-38bdf4a34198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:21.573015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7729a22-729c-42e2-957c-38bdf4a34198 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.573208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:21.573488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7729a22-729c-42e2-957c-38bdf4a34198 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.573881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7729a22-729c-42e2-957c-38bdf4a34198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-431fd3fd-9c6e-438f-80b6-61bb810d0f62 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:21.592029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:21.593215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:21.594225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:21.597489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 358/706] 10.222.0.39 () {56 vars in 1129 bytes} [Wed Jul 13 20:52:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:21.598941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c022b-54a4-427a-a221-a514884a8b9b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.599370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c022b-54a4-427a-a221-a514884a8b9b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.600335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c022b-54a4-427a-a221-a514884a8b9b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.602696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 359/707] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:21.603648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c022b-54a4-427a-a221-a514884a8b9b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:21.609127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:21.641711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:21.656785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7729a22-729c-42e2-957c-38bdf4a34198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:21.658378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7729a22-729c-42e2-957c-38bdf4a34198 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:21.743718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c022b-54a4-427a-a221-a514884a8b9b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:21.781488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:21.782205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.782588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.782989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 349/708] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:21.784510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.784924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.785344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.788474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.789337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:21.789750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:21.791289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:21.801622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 360/709] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Jul 13 20:52:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:21.802666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:21.805707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:21.806446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:21.880031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 350/710] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:21.881133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:21.891392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:21.909690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:21.941685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55df94e-a3c4-4e94-a0df-44437cb6af40 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:21.985203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:21.985991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:21.986271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:21.986544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:21.998848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.000717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:22.019567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:22.020502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:22.021373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:22.021373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:22.021373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:22.021373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:22.021373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 361/711] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:22.022933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.030635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd641cc4-5404-450f-9970-8e4388d8f7ad None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.067443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.068079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.068384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.068709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.069425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:22.069727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.071357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.077558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 351/712] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Jul 13 20:52:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:22.078398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.180674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:22.193968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21c29b-a91a-4b05-a7cb-a2e450fd249f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.204708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7672af-d015-4d75-8fbf-b820c82d2e32 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.233568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:22.244956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.245637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.245919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.246193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.246964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:22.247271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.249002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.264254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 352/713] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:22.272400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.273223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.273507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.273780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.274446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.275912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.276513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:22.295841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:22.296985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:22.297812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:22.305857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 362/714] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/services/network => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:22.307569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.403172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6374d63-0a00-4176-82c3-6803b866f2fc None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.463965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67596d-9a1f-4614-9100-138776135ec6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.468180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.469238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.470448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.470973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.482759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.484510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.487412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:22.507849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:22.508841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:22.509814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:22.510787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 353/715] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:22.515179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.516192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.516858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.518453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.518767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.520365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:22.520976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.524757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.535601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 363/716] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:22.541374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.594051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:22.646103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac613-77c3-4878-a447-bb17a59a88a9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.681847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.682533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.682866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.683148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.683862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] RBAC: Adding query filter params (type=network) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:22.684174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.685545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.693128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 364/717] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:22.729672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d0c38-2579-4d73-8a87-bd98fd8ad611 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.775777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.776497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.776786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.777063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.777868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:22.778183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.779691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503640de-adf3-463f-b858-8cd21b3ddff1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.798275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 354/718] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:22.804753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:22.947622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24920c83-bc7a-447c-acc3-10b71175e3ea None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:22.961549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:22.985287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:22.985941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:22.986219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:22.986489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:22.987372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a2048d28ca3b42d8b8d82f4754e44433) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:22.987681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:22.989126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e4a0f-dfad-4d10-a50b-3438788fff20 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:22.998000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 365/719] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:22 2022] GET /identity/v3/role_assignments?user.id=a2048d28ca3b42d8b8d82f4754e44433&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:23.127821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af596507-c8d4-4b0b-8834-8b28747447f3 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.128378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af596507-c8d4-4b0b-8834-8b28747447f3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.128378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af596507-c8d4-4b0b-8834-8b28747447f3 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.130033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 355/720] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:23.135779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cfc48-01af-4963-a9e4-a81618dc3945 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.136407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cfc48-01af-4963-a9e4-a81618dc3945 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.136912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cfc48-01af-4963-a9e4-a81618dc3945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.159697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e45cfc48-01af-4963-a9e4-a81618dc3945 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:23.161296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e45cfc48-01af-4963-a9e4-a81618dc3945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:23.237502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 366/721] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 20:52:23.264464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af596507-c8d4-4b0b-8834-8b28747447f3 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:23.361381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:23.429251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af596507-c8d4-4b0b-8834-8b28747447f3 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:23.456245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ec601-9eee-48b5-8066-9c68311a530b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.456550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ec601-9eee-48b5-8066-9c68311a530b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.456807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ec601-9eee-48b5-8066-9c68311a530b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.458473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 367/722] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:23.467994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.468272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.468555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.476382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:23.477485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.477980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.478459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.490484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:23.499116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:23.500104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:23.500969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:23.502827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:23.504062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 356/723] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:23.504170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:23.510907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:23.633541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 368/724] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:23.670408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1324fc5-61f3-4b25-ae11-e262e2d59b31 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:23.681351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:23.710918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:23.740374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:23.741102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.741325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.741598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.742385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:23.742671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:23.745028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:23.745500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:23.762416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 357/725] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:23.767856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:23.878389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24a2f9-31bb-4af1-b5d0-685614aaaa8e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:23.904142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e714df-6e54-4ff7-8b7b-3c2e79633c3c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:23.941477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:23.942264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.942541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.942836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.960208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:23.961175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:23.961459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:23.961728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:23.967519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:23.969226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:23.987200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:24.000923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:24.001839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:24.002720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:24.008848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 369/726] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:24.009391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:24.010237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:24.011167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:24.011167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:24.011167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:24.011167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:24.012238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 358/727] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:24.015565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.019956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.086976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:24.190808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:24.202429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facaebe5-d267-4fcb-ac11-be02426c65f1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.216551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0608a-af18-411d-8f07-e9dd8faa5f4d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.245045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:24.245721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.246003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.246277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.247033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:24.247340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:24.248704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:24.278497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 359/728] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:24.284841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.296872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:24.297561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.297848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.298128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.298919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:24.299227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:24.300617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:24.312150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 370/729] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:24.317755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.437296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a49471-f4d4-4d92-82cb-86fa34e6d9ba None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.481113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:24.481908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.482191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.482468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.490804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:24.499306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e0ebd-da75-4073-afb1-ebf7f1d21869 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.508459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:24.509927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:24.545787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:24.546529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.546827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.547105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-b4a20bd0-2d54-48c9-a7f1-434a7fabfb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:24.548701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:24.549524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:24.550388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:24.550388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:24.550388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:24.550388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:24.555418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 371/730] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:24.556534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded02b4f-32e4-4cf6-b6b2-5e1105196425 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.557042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded02b4f-32e4-4cf6-b6b2-5e1105196425 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.557577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded02b4f-32e4-4cf6-b6b2-5e1105196425 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.559317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 372/731] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:24.560190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded02b4f-32e4-4cf6-b6b2-5e1105196425 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.564310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:24.564645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:24.578012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ed5e9798-c36d-4c0e-a608-1fb6191596ab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:24.585907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:24.586878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:24.587811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 360/732] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:24.588897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.589006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.589257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.611532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:24.612726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:24.681951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded02b4f-32e4-4cf6-b6b2-5e1105196425 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.710640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 361/733] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:24.710640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.738716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:24.739725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.740764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.741293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.742221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:24.742762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:24.744328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:24.762022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 373/734] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:24.762904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:24.849262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:24.889669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2a92-fadf-4f1f-8630-283c154abbc1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.916615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:24.934176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58fac2-2e84-4300-82e5-3d81f1dae936 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:24.937980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:24.938694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:24.938983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:24.939261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:24.939998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:24.941361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:24.970531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 374/735] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:24 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:24.970531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.017077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.018223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.018738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.019260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.020337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:25.021071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.023649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.058666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 362/736] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:25.063737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.086883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ce22c-7288-4830-a598-97f8862def31 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.124942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.138669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.139503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.139785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.140060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.152148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.154382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-8ccfa233-e6ce-4c85-9bc3-07e2c79be765 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:25.175280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:25.176282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:25.177209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 375/737] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:25.178324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9e1f8-4aaf-4983-ada0-033ff8c84589 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.178324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9e1f8-4aaf-4983-ada0-033ff8c84589 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.178473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9e1f8-4aaf-4983-ada0-033ff8c84589 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.184401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 376/738] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:25.187827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9e1f8-4aaf-4983-ada0-033ff8c84589 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.215967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc580ec-91c0-4e14-9eb7-3fbfe814f64f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.259152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef481ad-5753-426f-a147-30dabde309e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.260547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef481ad-5753-426f-a147-30dabde309e9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.261301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef481ad-5753-426f-a147-30dabde309e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.262697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef481ad-5753-426f-a147-30dabde309e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.277569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.319913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9e1f8-4aaf-4983-ada0-033ff8c84589 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.354129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef481ad-5753-426f-a147-30dabde309e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.356117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef481ad-5753-426f-a147-30dabde309e9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.361563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.362308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.362691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.363056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.363967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:25.364716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.366239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bdb87-918e-467b-99b0-69a87d45278d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.385162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 377/739] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Jul 13 20:52:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:25.386248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.386535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.386864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.404361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:25.405537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:25.452973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.477214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 363/740] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:25.478562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc456-e58d-48ff-b3f6-65b9f7c8b01b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.478889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc456-e58d-48ff-b3f6-65b9f7c8b01b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.479166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfc456-e58d-48ff-b3f6-65b9f7c8b01b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.480543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 364/741] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:25.481124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc456-e58d-48ff-b3f6-65b9f7c8b01b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.486932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 378/742] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:25.488008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.526541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.588005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae23193-9e10-4ad0-b212-6d511c27ba0e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.593040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfc456-e58d-48ff-b3f6-65b9f7c8b01b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.621825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.622636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.622917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.623201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.628233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.628886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.629164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.629462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.633768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.635288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-9f9be40c-8506-4298-aed1-0e3779b01fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:25.652061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:25.653158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:25.654090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:25.654090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:25.654090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:25.654090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:25.654090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 379/743] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:25.654415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.654719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.654993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.671122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:25.672275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:25.696416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.707860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.709619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.739120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 380/744] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 20:52:25.739990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.763252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.830107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 365/745] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 20:52:25.831248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.852102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8201af-3a5d-4003-958b-7d17d7219fa4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.888559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.889260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.889540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.889822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.890539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.892157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.912661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 381/746] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:25 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:25.912661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:25.937640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:25.939119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bee6de-f5e1-496d-b3ef-2a3032fcfeff None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:25.984946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:25.985786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:25.986171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:25.986579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:25.987464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:25.987879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:25.989529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:25.998797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.005140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 366/747] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:26.006759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.036333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021ef78-05eb-4217-8aaf-dcd2ef71b5f1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.083415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.084273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.084554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.084830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.100456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:26.110159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:26.111239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:26.112308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 382/748] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:26.113168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.127973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5a85f-4ed9-4807-b6b1-928961911316 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.170946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.171679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.171963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.172244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.184175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.185637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:26.204853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:26.206014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:26.207106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:26.207106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:26.207106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:26.207106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:26.207106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 367/749] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:26.207106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.229180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.251244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af0e2c-7379-44fa-9fee-462010f672b7 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.257135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.296925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.298216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.299062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.299495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.309728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.311669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.317459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea57de5-8958-4707-92c1-b87ed466858e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:26.335880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:26.336994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:26.338066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:26.340263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 383/750] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:26.341110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.357913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.359095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.359752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.360176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.361544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:26.362021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.364088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.377557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 368/751] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:26.378697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.459789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfe345-757b-42ea-896e-96f049f63aca None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.497165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.497837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.498126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.498407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.499172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:26.499479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.501596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.513032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.514919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b92d61-3e3d-4bc2-9bb1-791b31e3b423 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.516828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 384/752] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:26.517723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.527476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.561536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.563322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.563788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.564210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.565170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:26.565625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.567750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.584979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 369/753] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:26.586666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.637671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd66304-3e2c-4cc8-b159-a8b9221681ed None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.675871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.677052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.677556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.678028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.689976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.691624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.692848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862e593-6335-4391-a838-10e6d9f5ab52 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:26.712347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:26.713446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:26.714409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:26.714409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:26.714409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:26.714409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:26.716313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 385/754] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:26.717237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.729794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.730397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.730727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.731008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.738288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.739898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:26.753822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:26.754753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:26.756213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 370/755] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:26.758140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.781580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.787036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:26.837498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966f4e8-3642-4988-9681-df2dfe19c10d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.861324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f4abc-4816-47d8-99c4-e25f3eb65bbd None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:26.875082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.876199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.876693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.877178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.878361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=4535c7d4c7a74095b9ccc6777ffee724) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:26.878983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.881193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.887941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 386/756] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/role_assignments?user.id=4535c7d4c7a74095b9ccc6777ffee724&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:26.889069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:26.897331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:26.898025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:26.898311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:26.898594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:26.899370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:26.899680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:26.901493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:26.913939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 371/757] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:26.919839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.012566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31493-84b5-49c3-b0d1-54a83980473b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.014303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:27.034727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:27.038321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07418cac-c944-4de4-8b28-9313348a1e4e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.055773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.057257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.057751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.058234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.059410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:27.059946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.061694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.076414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 387/758] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:27.083792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.093968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.094822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.095151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.095426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.103685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.105192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:27.124993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:27.125990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:27.126931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:27.134566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 372/759] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:52:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:27.144207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.232528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e67a5-bfe2-4354-9d64-45c7f5fcd757 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.280313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.280991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.281278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.281559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.282740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=292fed9f5de945c883cc1ffeb91da4cb) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:27.283063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.284613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f83246-7b2c-4cb6-bc69-b3f0e3aa2531 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.285366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a974b8-9c41-448b-aa76-4d3304bcafb8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.292421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 388/760] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/role_assignments?user.id=292fed9f5de945c883cc1ffeb91da4cb&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:27.293879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4821fd3-c2b7-4850-b9aa-3df3922adb31 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.294173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4821fd3-c2b7-4850-b9aa-3df3922adb31 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.294455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4821fd3-c2b7-4850-b9aa-3df3922adb31 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.296110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 389/761] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:27.301989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.302370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.303267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.327337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:27.329121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:27.335701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.336609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.337090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.337545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.338794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:27.339328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.341173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.362768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 373/762] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:27.370663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.378897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:27.395996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:27.492660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0483-66d0-45f2-8fcf-5dc7c7470aba None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.501260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 390/763] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:27.523020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.556970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.557698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.557984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.558267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.575044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.576500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:27.615101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:27.616738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:27.617762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:27.617762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:27.617762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:27.617762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:27.617762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 374/764] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:27.622254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.703959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b97c2c-3336-43dd-acf0-52c4d064cf3f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.726960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:27.770159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.770988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.771830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.772112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.772774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.774141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:27.790053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:27.791212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:27.792223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:27.793654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 391/765] 10.222.0.39 () {56 vars in 1129 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:27.805107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ab94c-d389-461f-8917-1a3f5e3db9f5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:27.808910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.868528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:27.869196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:27.869476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:27.869750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:27.870481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:27.870825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:27.872189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:27.874973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:27.890408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 375/766] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:27.899775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:27.993657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75018193-eef2-46d6-81d6-c9922f2d2c69 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.035433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.036100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.036387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.036667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.037420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:28.037729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.039906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.045668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 392/767] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:28.051125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.106660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e7e8-c489-48e8-90a2-1bf44e4ca0ec None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.143415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:28.180332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.181157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.181420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.181694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.182468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:28.187697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e4ca5-08e2-4280-b45f-d78f288de23a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.194686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.196145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:28.220087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:28.221035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:28.221921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:28.222827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 376/768] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:28.228810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.264200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.264922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.265206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.265483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.266243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:28.266554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.269425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.277121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 393/769] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Jul 13 20:52:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:28.282892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.415850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cffc6e5-aafe-4832-b0c3-e39d00f9f796 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.437233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df6fc6-b748-44c7-b0ab-0de090e9dfb1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.459537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.460681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.461114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.461766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.463304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:28.463901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.466380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac22ca11-ca1e-4985-ba9f-c8aaced7aef5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.473318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.474037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.474930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.475269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.476129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] RBAC: Adding query filter params (interface=public, service_id=1ed47a33fb524c03b83f82c5046eeb4c, region_id=RegionOne) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:28.476626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.478037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53703c98-ef8e-4cf2-9d1d-e574503d1cf6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.482719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 377/770] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:28.488037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f7b7f-3ba6-40b2-b76a-2803f9ffcaca None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.488320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f7b7f-3ba6-40b2-b76a-2803f9ffcaca None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.488594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f7b7f-3ba6-40b2-b76a-2803f9ffcaca None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.490589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 378/771] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:28.492339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f7b7f-3ba6-40b2-b76a-2803f9ffcaca None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.494464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 394/772] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:52:28 2022] GET /identity/v3/endpoints?service_id=1ed47a33fb524c03b83f82c5046eeb4c&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:28.499216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.499503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.499780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.517161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:28.518359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:28.541596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:28.553655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:28.610663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 395/773] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:28.611923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.637708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f7b7f-3ba6-40b2-b76a-2803f9ffcaca None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.678254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.679454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.679454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.679712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a2048d28ca3b42d8b8d82f4754e44433/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.681786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=a2048d28ca3b42d8b8d82f4754e44433, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.683304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.701140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 379/774] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:28 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a2048d28ca3b42d8b8d82f4754e44433/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:28.706883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.781598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac208a-e7e2-40b2-b134-3f4dc7e64053 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.869018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:28.872198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.873036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.873372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.873744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.874507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.874812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b1146-46f4-4eb3-bbff-22da5a7ebacb None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:28.876548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.883053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 396/775] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:28 2022] GET /identity/v3/services => generated 772 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:28.922063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:28.922919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:28.923203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:28.923476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] PATH_INFO: `/v3/services/network` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:28.924148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:28.925554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:28.940723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:28.941716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:28.942527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 380/776] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:28 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:28.949412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:28.976401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:29.075947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f888c-d264-4f76-9911-f7dd73ca4f11 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.112996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:29.113672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.113962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.114240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.115036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] RBAC: Adding query filter params (name=network) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:29.115346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:29.116723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e759d-0527-4a79-a494-b64ed3f17986 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:29.122400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 397/777] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:29.127913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:29.236159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd234f1f-9d2e-46ce-a766-90fd7c97d554 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.272176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:29.272853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.273137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.273413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.274138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] RBAC: Adding query filter params (type=network) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:29.274436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:29.275846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:29.281430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bcd48-03f4-4610-9116-887872f52429 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.281903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bcd48-03f4-4610-9116-887872f52429 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.281983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bcd48-03f4-4610-9116-887872f52429 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.283599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 398/778] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:29.284809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 381/779] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:29.290467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:29.290735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.291007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.291291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.324089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:29.328130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:29.329344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:29.409966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6e6a2-37f3-4713-a97d-1ced2b04e45f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.466826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 399/780] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:29.476795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:29.477526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.477812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.478136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.478973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] RBAC: Adding query filter params (interface=public, service_id=c81cbd61759d4a65a4bb9b28e73408f2, region_id=RegionOne) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:29.479291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:29.480662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:29.493939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 382/781] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:52:29 2022] GET /identity/v3/endpoints?service_id=c81cbd61759d4a65a4bb9b28e73408f2&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:29.499207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:29.516311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:29.519067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:29.638074 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:29.639269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79209dd-b2b0-4a48-a62a-28d4922225aa None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.649532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d4f37-2019-4ae9-a8ed-53f6aa7f754b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.675399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b95a3-be84-46a9-9cee-84cebbaa3868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:29.676395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b95a3-be84-46a9-9cee-84cebbaa3868 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.676881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b95a3-be84-46a9-9cee-84cebbaa3868 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.677416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b95a3-be84-46a9-9cee-84cebbaa3868 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.678840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b95a3-be84-46a9-9cee-84cebbaa3868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:29.680402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b95a3-be84-46a9-9cee-84cebbaa3868 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:29.686770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 400/782] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:29 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:29.688294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769a51-8ceb-4968-9acb-f6c074ae4cc4 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.688805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769a51-8ceb-4968-9acb-f6c074ae4cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.689310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db769a51-8ceb-4968-9acb-f6c074ae4cc4 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.690912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 401/783] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:29.692778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:29.693800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.694089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.694369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.698660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.698660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.698660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.715527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:29.717060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:29.720902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:29.738958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:29.740539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:29.741376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:29.741999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 383/784] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:29.746858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:29.795011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 402/785] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:29.822582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:29.839398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:29.945366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c724a94-3fb9-4af5-85f9-c1a5e71b5e2a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.972772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f7cea-4315-4022-ad9f-e8c3922ccea0 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:29.991406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:29.992554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:29.993152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:29.993501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:29.994382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:29.994808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:29.996251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.006790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 384/786] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:30.012813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.014374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.015440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.015903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.016446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.030839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:30.031931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:30.042563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:30.043510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:30.044353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:30.047322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 403/787] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:30.054366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.136636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a5a6f-04dd-4218-a0a0-56658871f897 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:30.142215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:30.175516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.176294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.176580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.176856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.185257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c378466-df13-4928-89e3-4db09ed87d55 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:30.188177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:30.189743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:30.208502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:30.209934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:30.211415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:30.211415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:30.211415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:30.211415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:30.211415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 385/788] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:30.216048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.233720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.234663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.234957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.235232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.235987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:30.237757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:30.238945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.260006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 404/789] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:30.267497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.337922 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:30.380430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca5f38-8fd2-4896-9823-a7205f34b827 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:30.446004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.446694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.446980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.447256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.447983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:30.448289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:30.449656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.466946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 386/790] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:30.468194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.474253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092a9b8-1395-46f6-8720-0008e7be7bdf None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:30.504974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:30.513496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.514893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.515435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.515909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.528790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:30.530586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:30.549712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:30.550664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:30.551567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:30.551567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:30.551567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:30.551567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:30.554351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 405/791] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:30.561254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.690667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99199ea9-ca6c-4ca2-b96c-4247ae6bad7e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:30.708354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd87907-fdeb-49a2-8835-80d9a2607555 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:30.741144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.741829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.742116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.742389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.743252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:30.743553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:30.745033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.750242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:30.763382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 406/792] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:30.769417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.773517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:30.775341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:30.775782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:30.780070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:30.792592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:30.795678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:30.824038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:30.825223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:30.826365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:30.830475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 387/793] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:30.839993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:30.854983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:30.950424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd7da78-66c4-461c-9360-ebda8389c00e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.011595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.012665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.013258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.013686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.022377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.024000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.025657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb85263-7312-445c-ad9b-b1762aa4482f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:31.050771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:31.051943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:31.053113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:31.057614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 407/794] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:31.064933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.088424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.089104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.089391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.089671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.090518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:31.090858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.092284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.126253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 388/795] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Jul 13 20:52:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:31.132347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.187206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.215697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b0478-4464-4273-a0a8-022134fee1bf None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.218277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.247508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e849e5e-8335-4d42-889b-0aed1d4cc277 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.253658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.254350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.254664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.254955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.256493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:31.256817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.258310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad94daad-50e6-4ecf-a518-0f572c1667dc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.286544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 408/796] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:31.286544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ce39b-0775-4cb4-b1c3-f0a4f7258a8c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.286544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ce39b-0775-4cb4-b1c3-f0a4f7258a8c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.286544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ce39b-0775-4cb4-b1c3-f0a4f7258a8c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.289800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 409/797] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:31.290726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ce39b-0775-4cb4-b1c3-f0a4f7258a8c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.300475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.301537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.302020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.302539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.313386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.315088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-cbd0528a-0ced-4a2b-a562-f5622e269808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:31.334437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:31.335387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:31.336263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:31.336263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:31.336263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:31.336263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:31.338571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 389/798] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:31.340032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.340526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.341029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.358063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:31.359631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:31.399691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ce39b-0775-4cb4-b1c3-f0a4f7258a8c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.427156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 390/799] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:31.427858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.447360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.448995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.449693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.450294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.452161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.462379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.463872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-23ac3ced-32f3-49c3-a86a-01ce47005850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:31.481005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:31.482313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:31.483257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:31.483257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:31.483257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:31.483257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:31.483257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 410/800] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:31.483529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fa7a9-468b-4359-9031-d30f26b2b2d8 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.484030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fa7a9-468b-4359-9031-d30f26b2b2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.484030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fa7a9-468b-4359-9031-d30f26b2b2d8 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.489227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 411/801] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:31.490012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fa7a9-468b-4359-9031-d30f26b2b2d8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.503994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.540650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890d93c-3556-4a81-b7f2-983aac0ac8cf None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.578662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.579585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.580052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.580509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.581428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:31.581936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.583407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.595297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 391/802] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:31.595991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fa7a9-468b-4359-9031-d30f26b2b2d8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.596127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.634551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e693a-40b4-49c9-9c7a-2d2138b3813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.635364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e693a-40b4-49c9-9c7a-2d2138b3813f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.635653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e693a-40b4-49c9-9c7a-2d2138b3813f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.635932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e693a-40b4-49c9-9c7a-2d2138b3813f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.647638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e693a-40b4-49c9-9c7a-2d2138b3813f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-404e693a-40b4-49c9-9c7a-2d2138b3813f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:31.655266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:31.656173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:31.657045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 412/803] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:31.657736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.658412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.658412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.675531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:31.676722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:31.694856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.712664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ecb14-c8cc-4fa2-9995-d3f0c36018f6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.747244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 413/804] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:31.748328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.750642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.758990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.759947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.760933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.761407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.762769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:31.763511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.765354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.781793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 392/805] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:31.782562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.861234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5467795-9640-4219-a61b-34a98f0bbdbc None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.892631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdb229-7044-4af2-aa61-04d2acd85b07 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:31.898136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.898963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.899251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.899520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.906499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.908006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:31.925912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:31.926824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:31.928278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 414/806] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:31.928755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.933676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:31.934359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:31.934670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:31.934953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:31.935710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:31.936017 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:31.937380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:31.947522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 393/807] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:31.951552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:31.961565 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:31.998089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.042050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2aa3-d935-4b06-ad67-eb668f12afcb None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.064323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63c24f-d50b-47f9-bf6e-2cab8e01ce2d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.078026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.078860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.079145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.079423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.080080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.081447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:32.093908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:32.094890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 20:52:32.095723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:32.101063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 415/808] 10.222.0.39 () {56 vars in 1129 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:32.101783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.102728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.103575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.103925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.104272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.112589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.114360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:32.129839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:32.130835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:32.131753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 394/809] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:32.132259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.198394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.209271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb28624-32e6-44e9-b22f-9cb6631d71ef None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.233348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ee344-7348-4b60-9734-b88c77e78147 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.234016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.243782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.244439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.244720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.244989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.245771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:32.246074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.247525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.259166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 416/810] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:32.259932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.274364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.275302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.275586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.275855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.286320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.288612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:32.307313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:32.308253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:32.314052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:32.314052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:32.314052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:32.314052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:32.314052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 395/811] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:32.314052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.358209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc640e4-88db-4a4d-9312-18393073bdae None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.391065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.391650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.391932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.392207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.392910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:32.393213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.394566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f517b4-bfb6-46d8-ad35-59cea948de0d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.399454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 417/812] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:32.400613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc13e3e-11f9-4114-bca7-fdfecc6e5fae None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.400897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc13e3e-11f9-4114-bca7-fdfecc6e5fae None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.401168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc13e3e-11f9-4114-bca7-fdfecc6e5fae None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.402369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 418/813] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:32.403054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc13e3e-11f9-4114-bca7-fdfecc6e5fae None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.414219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff200c3-f60b-4088-80f2-190807a0182a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.431267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.448052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.448704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.448984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.449263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.450038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:32.450340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.451775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.452654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.464153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 396/814] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:32.464923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.502996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc13e3e-11f9-4114-bca7-fdfecc6e5fae None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.537646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.538818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.539100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.539378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/4535c7d4c7a74095b9ccc6777ffee724/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.553510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=4535c7d4c7a74095b9ccc6777ffee724, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.554966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.560655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920038cc-bf4e-4ee9-8357-1701b23d2090 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.566950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 419/815] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:32 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/4535c7d4c7a74095b9ccc6777ffee724/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:32.578737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.612242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.612920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.613202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.614042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.614869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:32.615179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.617001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ab922-32e3-4b21-9641-0d28c82a44e6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.631473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 397/816] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:32.632617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482370-0e84-4263-9763-32409c205745 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.632904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482370-0e84-4263-9763-32409c205745 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.633180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482370-0e84-4263-9763-32409c205745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.651436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5d482370-0e84-4263-9763-32409c205745 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:32.652580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5d482370-0e84-4263-9763-32409c205745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:32.684274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.687530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757404ec-3915-4f80-a113-684b17b1bd75 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.687929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.733263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.733949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.734231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.734507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.735046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 398/817] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:32.735463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:32.735633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.735737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482370-0e84-4263-9763-32409c205745 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.736967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.743263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 420/818] 10.222.0.39 () {56 vars in 1124 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:32.744067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.843290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482370-0e84-4263-9763-32409c205745 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.847482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf7c21-91fa-478c-a89a-f0abc705eb47 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:32.879512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.880923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.881203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.881474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/292fed9f5de945c883cc1ffeb91da4cb/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.882306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:32.883117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:32.883396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:32.883494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=292fed9f5de945c883cc1ffeb91da4cb, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.883668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:32.884959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.891118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:32.892574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:32.900277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 399/819] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:32 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/292fed9f5de945c883cc1ffeb91da4cb/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:52:32.900928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:32.919199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:32.920178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 421/820] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:32.921225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:32.944025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:32.948994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:33.050222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d4e39-bd15-4a62-b674-375d7c07d6e4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.055107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd8688-728e-4c99-b199-666035407242 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.090679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.091753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.092037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.092312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] PATH_INFO: `/v3/services/network` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.092969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.094896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.102853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.103848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.104345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.104827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.105769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.107364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:33.107610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:33.108540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:33.109378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 422/821] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:32 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:33.110185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:33.142228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 400/822] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:33.143624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:33.249797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:33.253771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:33.277339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1b46-1e6b-4c9f-bbb5-8b7c6f3d9263 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.280008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f83de-bcaa-4c25-801f-7af579435b4a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.323736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.327569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.328894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.329551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.331765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:33.332325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.335272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.344266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 401/823] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:33.349561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.351005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:33.352805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.355188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.355734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.356848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:33.358565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.363335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.409014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 423/824] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:52:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:33.420839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:33.470936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aea96-960e-474b-91f9-53d9a74261e4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.508031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.509112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.509645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.510140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.511081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] RBAC: Adding query filter params (type=network) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:33.511589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.513150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.518955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 402/825] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:33.524534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:33.530738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446349b6-7078-449a-a51d-a2eab8adfdd7 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.538328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:33.569210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.569862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.570146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.570424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.590992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:33.637833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bd892-00ac-4a61-bc66-beb1c2ca5fbc None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.645074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.646782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.674194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.675213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.675702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.676178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.677181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.679298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.700861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 403/826] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:33.778239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 424/827] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:52:33.783699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:33.863264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:33.914236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee95d6-981b-442f-bc69-1f6fa2d43bd8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:33.949401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:33.950250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:33.950847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:33.951131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:33.962662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:33.964183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:33.982225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:33.983178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:33.984073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:33.984073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:33.984073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:33.984073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:33.984073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 404/828] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:33.990783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:34.045974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:34.103551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f45e1-a384-46cc-b795-f8bd4485aa41 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:34.142973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:34.143640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:34.143925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:34.144201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:34.144930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:34.145236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:34.147199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:34.159616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 425/829] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:34.167949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:34.290072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012defc0-8cdb-458e-b0a2-54fbe75ef758 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:34.307056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:34.326720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:34.327526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:34.327808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:34.328465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:34.335796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:34.337267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:34.351722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:34.352581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:34.353535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 405/830] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:34.360282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:34.463672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef23965-703e-4846-b226-dd587fe5b78e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:34.474936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:34.498736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:34.499407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:34.499695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:34.499976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:34.500774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:34.501149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:34.502767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6f545-c9ae-472b-a87b-4df2dc3061c1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:34.516101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 426/831] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:34.522457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:34.626208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb7648-56b8-4084-bc80-fe8b2d54a564 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:34.663930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:34.664970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:34.665436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:34.665888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:34.667004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a2048d28ca3b42d8b8d82f4754e44433) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:34.667496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:34.669184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e3350-530d-45f4-8e1b-d7bb8ec82a4f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:34.676528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 406/832] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:34 2022] GET /identity/v3/role_assignments?user.id=a2048d28ca3b42d8b8d82f4754e44433&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:34.913997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa2941-7894-4b0d-968c-35e2110a0531 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:34.915020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa2941-7894-4b0d-968c-35e2110a0531 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:34.915560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa2941-7894-4b0d-968c-35e2110a0531 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:34.917411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 427/833] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:34.922870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a27ce0-7d0a-447c-99f9-cc32297f1a99 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:34.923723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a27ce0-7d0a-447c-99f9-cc32297f1a99 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:34.924203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a27ce0-7d0a-447c-99f9-cc32297f1a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:34.941907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-69a27ce0-7d0a-447c-99f9-cc32297f1a99 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:34.943462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-69a27ce0-7d0a-447c-99f9-cc32297f1a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:34.973275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.014112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 407/834] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:35.038251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa2941-7894-4b0d-968c-35e2110a0531 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.118438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071b19b-3c89-4885-b940-b0192e8bd440 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.118790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071b19b-3c89-4885-b940-b0192e8bd440 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.119025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071b19b-3c89-4885-b940-b0192e8bd440 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.120727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 408/835] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:35.126731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.127226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.127506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.148278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:35.149772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:35.152399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa2941-7894-4b0d-968c-35e2110a0531 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.197339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.198068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.198351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.198648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.210108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:35.217798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:35.218775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:35.219615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 428/836] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:35.225633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.242718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 409/837] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:35.259028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.263629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.358983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c09877-928c-45cb-aeb5-c1c6314b390f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.377504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.385353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40f169-dbc7-4b1b-9c87-0cffbd72b38c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.408837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.409808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.410287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.411295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.413318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:35.413864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:35.416432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:35.426427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.427441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.427790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.428134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.428899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 429/838] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:35.434062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.444734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:35.452505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:35.453437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:52:35.454268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:35.459071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 410/839] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:35.459937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.559238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.568885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acbc9f-3447-45b0-a37b-6c1157354691 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.585066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eafd0-fc33-42e0-9001-e98c2592c721 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.607552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.608219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.608500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.608779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.609934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:35.610500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:35.612062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd14fb1-be35-4a79-96d7-6bc0e197719f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:35.621223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 411/840] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:35.622580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547e851-c1f1-4adc-a301-5e2032470c57 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.622995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547e851-c1f1-4adc-a301-5e2032470c57 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.623374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547e851-c1f1-4adc-a301-5e2032470c57 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.626663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 412/841] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:35.626663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547e851-c1f1-4adc-a301-5e2032470c57 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.629681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.631129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.631440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.631720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.642730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:35.644183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:35.649183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-5b7a2e63-daf6-4722-8fd6-5d6a0364c0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:35.663313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:35.664177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:35.665115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:35.665115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:35.665115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:35.665115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:35.665115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 430/842] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:35.666087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.666087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.666332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.683242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:35.684426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:35.747461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547e851-c1f1-4adc-a301-5e2032470c57 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.756076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 431/843] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:35.756810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.795529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.796407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.796762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.797110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.808435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:35.810486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:35.829552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:35.832953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:35.834707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:35.840704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:35.840704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:35.840704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:35.840704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:35.840704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 413/844] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:35.840704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.872507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f9da-ecd9-42d5-b956-b16bbea91c8c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.910278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.910973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.911256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.911569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.912292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:35.912846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:35.913460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:35.914656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:35.930787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 432/845] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:35.931626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:35.953097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce678a-c1e9-4b81-8ea9-7d32835e4cf7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:35.988702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:35.989518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:35.989810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:35.990081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:35.990765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:35.992131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:36.004705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:36.006195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:36.007048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 414/846] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:36.007856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.045491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa5b8c-1873-4dea-a62e-97b5815fd96f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.076301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.083866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.084528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.084814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.085093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.085815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:36.086127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.087520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.099449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 433/847] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:36.100225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.118316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166c713-25d4-48b2-8adf-01cdcbd5c0a6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.153439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.154250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.154537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.154845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.157183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.164270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.166275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:36.182668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:36.183610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:36.184532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 415/848] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:36.185132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.216189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc506a-c405-4f20-8b1f-9bce272ca5c1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.251174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.252109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.252606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.252883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.253593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:36.253898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.255856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.260739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 434/849] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:36.261511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.295698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c31f2-d728-4897-a173-1e018c54a571 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.328449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.331500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.332315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.332581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.332963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.342235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.343707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:36.358555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:36.359647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 416/850] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:36.360657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.379858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ff28-95fd-48e5-bdec-c6e1eb42b2ce None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.415255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.415921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.416201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.416475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.417311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:36.417617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.419071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.430857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.438356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 435/851] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:36.439034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.482359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828e076-b7b5-4c79-8dc6-71003e7332b9 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.519863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.520416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.520697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.521574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.522288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:36.522591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.523962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.528652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 417/852] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:36.529260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.555375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73732b3-1849-4ece-9310-d4603321fba3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.571726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.595125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.595804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.596198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.596472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.597447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:36.597755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.599357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.619715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 436/853] 10.222.0.39 () {56 vars in 1135 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:36.625854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.668161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fef67-50d5-434e-8c0c-eba40788e2ea None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.691469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.721080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d9fdc-d1bd-439e-b6c6-f2062aea27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.721746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d9fdc-d1bd-439e-b6c6-f2062aea27ef None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.722029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d9fdc-d1bd-439e-b6c6-f2062aea27ef None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.722304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d9fdc-d1bd-439e-b6c6-f2062aea27ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.760639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc202f9-2fe8-48d2-8c52-9a8b60712524 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:36.800250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9febafc-7a35-4ba1-8b67-44d76e36171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:36.800831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9febafc-7a35-4ba1-8b67-44d76e36171a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.801122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9febafc-7a35-4ba1-8b67-44d76e36171a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.801406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9febafc-7a35-4ba1-8b67-44d76e36171a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.809820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d9fdc-d1bd-439e-b6c6-f2062aea27ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.811316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d9fdc-d1bd-439e-b6c6-f2062aea27ef None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.843331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.877937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9febafc-7a35-4ba1-8b67-44d76e36171a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:36.879413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9febafc-7a35-4ba1-8b67-44d76e36171a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:36.921538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 418/854] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:52:36.921538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f797-0cfc-4d09-a385-2c210b024663 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.921538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f797-0cfc-4d09-a385-2c210b024663 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.921538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95f797-0cfc-4d09-a385-2c210b024663 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:36.921538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 419/855] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:36.921538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f797-0cfc-4d09-a385-2c210b024663 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:36.942206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:36.992612 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 437/856] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:52:36.993940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:36.994233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:36.994513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.014723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:37.016297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:37.043137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95f797-0cfc-4d09-a385-2c210b024663 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.081169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.081724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.083072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.083557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.084011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.094640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 438/857] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:37.094640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.102446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.103925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:37.122073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:37.123019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:37.123871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:37.123871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:37.123871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:37.123871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:37.123871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 420/858] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:37.124652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.195779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.200346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79279e56-8b65-401a-b35b-79eeb4210116 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.227838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce29cf-7f6b-4b15-98ea-cdc78718b470 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.235238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.236036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.236321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.236599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.247544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.249015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.264261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.265395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.265904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.266440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.267315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:37.268174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:37.269042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:37.269929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:37.269929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:37.269929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:37.269929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:37.271837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.274036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 439/859] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:37.274982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:37.290140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:37.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:37.292042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 421/860] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:37.292891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.396102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f918e-03fa-4ecf-a5de-e20b7026bf5f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.432556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.433228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.433514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.433795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.434532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:37.434869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.436252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.437849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.453488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 422/861] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:37.454420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.460400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.509335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d59a0e-d8fa-4f74-8b6a-dd15e1b0c810 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.544835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.545716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.546148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.546560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.547486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:37.547952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.549466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.561130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 440/862] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:37.561801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116a427-29e1-497e-8869-02ed9a4a237c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.562236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.599793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.600455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.600741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.601016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.601734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:37.602042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.603434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.613113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 423/863] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:37.613874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.670024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198fc15-a55e-4277-a292-ab16354de68e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.684117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.686558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.709883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.710711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.710998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.711274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.718173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.719652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.732697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fce04b-03da-47c1-ad0d-7973eacefe9a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:37.734778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:37.735640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:37.736517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 441/864] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:37.736992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.773824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.774991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.775480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.775970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.785369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.788228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:37.802640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:37.803672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:37.804675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:37.806704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 424/865] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:37.807612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.843534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921546c4-89c3-43b8-817f-2105d53c33dd None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.877450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.878118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.878397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.878700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.879411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:37.879720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.881113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:37.890655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 442/866] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:37.890655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:37.926985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.934719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033d87-eafe-4ece-96ed-6b3cfbba3580 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:37.960028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:37.979620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:37.981985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:37.982733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:37.983246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:37.984257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:37.984759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:37.986453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:38.004521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e17a3-abd6-4bd1-a9fc-f6bd3a3ec919 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:38.006151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 425/867] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:38.009179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:38.046785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:38.047477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.047764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.048050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.048854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:38.049174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:38.050608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:38.064885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 443/868] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:38.071424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:38.150356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa69d10-6e93-4954-9fc1-a04d0d907a1f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:38.178782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:38.185214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37eff3-cf77-466b-8b1a-2a8243628a65 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:38.189581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:38.190500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.191072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.191557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.193166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=292fed9f5de945c883cc1ffeb91da4cb) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:38.193758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:38.195411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd096983-bdf8-4ef8-b331-9eae76c7be6f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:38.202746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 426/869] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:38 2022] GET /identity/v3/role_assignments?user.id=292fed9f5de945c883cc1ffeb91da4cb&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:38.223999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:38.224734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.225029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.225310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.226179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=4535c7d4c7a74095b9ccc6777ffee724) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:38.226499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:38.228030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3803bd0-e86a-4e75-8da3-8539a5d019af None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:38.235747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 444/870] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:38 2022] GET /identity/v3/role_assignments?user.id=4535c7d4c7a74095b9ccc6777ffee724&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:38.616574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37a068-5656-4eab-8a22-c3545ca6d8bd None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.616997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37a068-5656-4eab-8a22-c3545ca6d8bd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.617173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37a068-5656-4eab-8a22-c3545ca6d8bd None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.618823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 427/871] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:38.624050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56626e84-ff7e-4307-bd5d-428018b1f464 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.624347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56626e84-ff7e-4307-bd5d-428018b1f464 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.624633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56626e84-ff7e-4307-bd5d-428018b1f464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.642866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-56626e84-ff7e-4307-bd5d-428018b1f464 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:38.644071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-56626e84-ff7e-4307-bd5d-428018b1f464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:38.714238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 445/872] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:38.737241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37a068-5656-4eab-8a22-c3545ca6d8bd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:38.784737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:38.850375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37a068-5656-4eab-8a22-c3545ca6d8bd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:38.885435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3176f5-eac7-49a2-9788-41625c57e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:38.886481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3176f5-eac7-49a2-9788-41625c57e6e0 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:38.886999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3176f5-eac7-49a2-9788-41625c57e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:38.887467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3176f5-eac7-49a2-9788-41625c57e6e0 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:38.888392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3176f5-eac7-49a2-9788-41625c57e6e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:38.889995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3176f5-eac7-49a2-9788-41625c57e6e0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:38.906425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 428/873] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:38 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:39.953144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689f506-053a-4030-bace-b73a3a508f40 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:39.953666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689f506-053a-4030-bace-b73a3a508f40 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:39.953743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689f506-053a-4030-bace-b73a3a508f40 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:39.955367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 446/874] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:39.960437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8fc88-2ed0-4943-9cd3-0703fdf2bcd9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:39.960853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8fc88-2ed0-4943-9cd3-0703fdf2bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:39.961019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8fc88-2ed0-4943-9cd3-0703fdf2bcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:39.979121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a7e8fc88-2ed0-4943-9cd3-0703fdf2bcd9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:39.980315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a7e8fc88-2ed0-4943-9cd3-0703fdf2bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:40.065585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 429/875] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:40.087984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689f506-053a-4030-bace-b73a3a508f40 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.185433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:40.196592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689f506-053a-4030-bace-b73a3a508f40 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:40.231765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c098d-6449-45c8-beab-f467003da8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:40.232360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c098d-6449-45c8-beab-f467003da8a6 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.232645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c098d-6449-45c8-beab-f467003da8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.232928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c098d-6449-45c8-beab-f467003da8a6 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.233669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c098d-6449-45c8-beab-f467003da8a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:40.235074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c098d-6449-45c8-beab-f467003da8a6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:40.251411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 447/876] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:40 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:40.346079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e5335-ffc3-4604-b88e-b213ab5bf0c0 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.346536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e5335-ffc3-4604-b88e-b213ab5bf0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.346956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e5335-ffc3-4604-b88e-b213ab5bf0c0 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.349525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 430/877] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:40.355542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.356009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.356390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.375592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:40.376873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:40.436694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7f541-34f1-4565-82e9-2b76b82b6de9 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.437105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7f541-34f1-4565-82e9-2b76b82b6de9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.437395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7f541-34f1-4565-82e9-2b76b82b6de9 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.439172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 431/878] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:40.445190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.445499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.445787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.456323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 448/879] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:40.467356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:40.468553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:40.479236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.493859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:40.543038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 432/880] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:40.565816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.575380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:40.603333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1a079-76bb-4531-9777-7fed53aaeb46 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:40.637777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:40.638712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.639072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.639422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.640159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:40.641708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:40.654728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:40.655765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:40.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 449/881] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:40.662661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.687471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a7cb-8a7d-4d2d-a7f9-ce8934845783 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:40.726183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:40.727036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.727322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.727596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.739389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:40.748019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:40.748923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:52:40.749745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:40.757221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 433/882] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Jul 13 20:52:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:40.758376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.781772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:40.785212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c435a-a545-459b-96cc-22884dc24ca8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:40.824539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:40.825209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.825482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.825771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.826457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:40.826792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:40.828160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:40.838304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 450/883] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:40.839606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.843100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:40.875536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e997a-81f2-4567-b399-85f357d6a4e1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:40.912005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:40.913270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.913780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.914271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.915248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:40.915771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:40.917411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:40.931093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 434/884] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:52:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:40.932239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:40.945904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711ca40-5919-4654-860c-7304d76bbace None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:40.980939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:40.981608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:40.981885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:40.982163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:40.982912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:40.983270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:40.984643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de56392-491c-4fa3-8f4d-01eac98464b7 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:40.994935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 451/885] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:41.015645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f4002-488b-4d70-96de-3aaa40bf8961 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.015938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f4002-488b-4d70-96de-3aaa40bf8961 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.016174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f4002-488b-4d70-96de-3aaa40bf8961 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.022744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 452/886] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:41.024629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.025028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.025454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.029650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.044359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:41.045738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:41.053135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da780c09-b818-408e-be24-a85d3a3ccc54 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.091830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.092898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.093332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.093788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.105998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.107749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.120470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:41.131931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:41.132974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:41.133871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:41.133871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:41.133871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:41.133871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:41.134099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 453/887] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:41.136300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 435/888] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:41.142111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.154469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.253049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908b256-a2fe-4abb-900e-9206e1f21c73 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.285593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f18f-078a-439e-8e9f-a428e29e7e36 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.291388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.292384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.292894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.293368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.294293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:41.294827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.296398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.313452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 454/889] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:41.314589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.322070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.322952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.323234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.323512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.324170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.325524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:41.338250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:41.339319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:41.340175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 436/890] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:41.342955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.346276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.382342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.426090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca151e-39a6-46fa-a09c-2d88f9b7bf93 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.473796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.474937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.475308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0a880-998c-4f41-acd3-62891bc0526c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.475545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.475980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.487531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.489045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:41.511363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:41.512347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:41.513261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:41.516169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 455/891] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:41.521707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.523705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.524381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.524665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.524944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.525663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:41.526104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.528021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.537122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 437/892] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:41.538383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.601876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.652176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234d798-a282-487f-ab14-a7da97a8e8ce None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.659575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0460ac-1f54-4312-b08d-72e55f67fc46 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.679499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.695990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.696840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.697231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.697678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.698441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:41.698830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.700253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.703810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.704670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.705128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.710130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.710270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 438/893] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:41.710352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:41.710352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.710352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.711177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.734222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 456/894] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:41.735360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.831361 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df501375-746c-41b2-90ea-fe154a18078f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.856044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.860556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27412b-d9cb-44d8-a303-4859e0dad02c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.869889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.870849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.871249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.871679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.872571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] RBAC: Adding query filter params (interface=public, service_id=f75c3bb2b59b411d8cd42b1743ca9fba, region_id=RegionOne) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:41.873007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.874549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:41.884029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 439/895] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/endpoints?service_id=f75c3bb2b59b411d8cd42b1743ca9fba&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:41.885026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:41.910410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:41.912318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:41.913129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:41.913649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:41.963723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:41.990526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:41.991345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c713b49-2b00-49b2-b695-37ffe727adfe None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:41.993075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.027757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.028429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.028714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.029151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.029845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.031246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.044688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 440/896] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:52:42.112454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 457/897] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:41 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:52:42.118378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.181904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.234091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c11f43-3cb4-439c-a809-291978eaeedb None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.271622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.272431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.272720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.272998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.284657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.286178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-f7412855-0101-478a-bcc0-ad335d68ebda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:42.304391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:42.305400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:42.306273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:42.306273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:42.306273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:42.306273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:42.306273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 441/898] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:42.311197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.390784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.410728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e14a-dc17-48ee-8001-fba712111ed6 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.411790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e14a-dc17-48ee-8001-fba712111ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.412096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2e14a-dc17-48ee-8001-fba712111ed6 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.414705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 442/899] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:42.419647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.419932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.420212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.422136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328b6f-cc3f-492d-8170-0b7b5d495bc9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.437520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:42.438902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:42.458078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.459271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.459554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.459880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.460603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:42.460906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.462368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.475107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 458/900] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:42.481272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.530637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 443/901] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:42.546891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.591623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.613020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9117-c981-4d55-93a1-cbf3e0ec1c8d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.653493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.654609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.655111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.655584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.658913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.665694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.667462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.676026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47032e4-804f-4dab-a246-baea2fdfef36 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:42.682350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:42.683247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:42.684112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:42.691198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 459/902] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:42.692264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.717861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.718954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.719255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.719587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.720245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.722387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:42.738705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:42.739732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:42.740550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:42.745486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 444/903] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:42.746838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.808281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f0aa8-b7d4-4e39-bb93-fdd3752626fb None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.851389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.852057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.852384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.852663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.853889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:42.854218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.856149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.878005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 460/904] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:42.879360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.880596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:42.886727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a833-4e04-46a0-8f9a-88a30c349588 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:42.922301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:42.923100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:42.923479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:42.923890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:42.924674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:42.925249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:42.926560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:42.936637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 445/905] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:42.938605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:42.962162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:43.023427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a7972-cdfd-4143-a2a5-8734cbd21069 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.052608 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb053a8-6847-4df0-a2e2-f8fa2ae33af2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.071728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.072743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.073223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.074206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.075604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] RBAC: Adding query filter params (role.id=82ae094868e64fd49c4f9c9333153a10, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=4535c7d4c7a74095b9ccc6777ffee724) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:43.076121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.078455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb47f94-442d-4f92-a811-928ab45e2e7a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.087147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 461/906] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/role_assignments?user.id=4535c7d4c7a74095b9ccc6777ffee724&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=82ae094868e64fd49c4f9c9333153a10 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:43.095467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.096447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.096877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.097299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.098180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:43.098755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.100394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.104860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21dd8-1da0-4ace-a83c-986113d0e98d None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.105171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21dd8-1da0-4ace-a83c-986113d0e98d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.105413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f21dd8-1da0-4ace-a83c-986113d0e98d None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.107797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 462/907] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:43.113722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.114056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 446/908] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:43.115086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.115583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.120420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.140534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:43.142404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:43.155578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:43.231325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 463/909] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:43.237816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2615fa-f8a0-440c-bd52-ed1da3ace9b7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.250283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.288271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.289183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.289673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.290123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.291240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] RBAC: Adding query filter params (interface=public, service_id=a8ecafcd3a0f49e8b832271f5191bf6e, region_id=RegionOne) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:43.291726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.293317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.304222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 447/910] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:52:43 2022] GET /identity/v3/endpoints?service_id=a8ecafcd3a0f49e8b832271f5191bf6e&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:43.305759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:43.319474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:43.385462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227667b7-957c-4a64-8e60-03472bde60b3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.420223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:43.421973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd03f87-1cc8-4560-97df-ba2a14c0a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.422872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd03f87-1cc8-4560-97df-ba2a14c0a730 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.423638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd03f87-1cc8-4560-97df-ba2a14c0a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.424120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd03f87-1cc8-4560-97df-ba2a14c0a730 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.425165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd03f87-1cc8-4560-97df-ba2a14c0a730 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.427367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd03f87-1cc8-4560-97df-ba2a14c0a730 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.437119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0290d5-4838-450a-a73d-7f66e2d58fce None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:43.451355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 464/911] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:43 2022] POST /identity/v3/services => generated 241 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:43.486915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cdb3d-bbf3-4a26-bd3f-4bc95fabeaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:43.487543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cdb3d-bbf3-4a26-bd3f-4bc95fabeaac None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:43.490650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cdb3d-bbf3-4a26-bd3f-4bc95fabeaac None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:43.490650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cdb3d-bbf3-4a26-bd3f-4bc95fabeaac None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:43.490650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cdb3d-bbf3-4a26-bd3f-4bc95fabeaac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:43.491091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cdb3d-bbf3-4a26-bd3f-4bc95fabeaac None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:43.498420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 448/912] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:43 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:52:44.160010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec02ef-a3f7-4799-b802-487045014151 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.160534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec02ef-a3f7-4799-b802-487045014151 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.160534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec02ef-a3f7-4799-b802-487045014151 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.162182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 465/913] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:44.167746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.168179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.168358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.186249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:44.187474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:44.269786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 449/914] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:44.293262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec02ef-a3f7-4799-b802-487045014151 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.403103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec02ef-a3f7-4799-b802-487045014151 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.414114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:44.438642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.439436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.439889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.440166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.440818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.442871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:44.454969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:44.455932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:44.456785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 466/915] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:44.464069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.558962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:44.568814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba9871-3ecc-4687-9048-35d23fc03be3 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.605482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.606167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.606450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.606757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.607488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:44.607796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.609329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.616374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 450/916] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:44.622365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.727520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0ae2c-b673-43b4-8b91-8797b16dd840 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.762075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.762762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.763048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.763326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.764035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:44.764340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.765710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940ffe0-59ec-4daf-bcaf-365cb8bb318b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.771074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 467/917] 10.222.0.39 () {56 vars in 1114 bytes} [Wed Jul 13 20:52:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:44.776515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:44.881461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655b3af-52f8-4369-a118-b730db7516a7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:44.921259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8ab5-0666-4cbc-8919-2fbb8aa0746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:44.921924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8ab5-0666-4cbc-8919-2fbb8aa0746c None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:44.922196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8ab5-0666-4cbc-8919-2fbb8aa0746c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:44.922472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8ab5-0666-4cbc-8919-2fbb8aa0746c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:44.923251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de8ab5-0666-4cbc-8919-2fbb8aa0746c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:44.924608 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de8ab5-0666-4cbc-8919-2fbb8aa0746c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:44.938951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:44.945735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 451/918] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:45.174001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde6ed0-65d4-4faa-8e29-28cd7053a3f6 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.174553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde6ed0-65d4-4faa-8e29-28cd7053a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.174553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde6ed0-65d4-4faa-8e29-28cd7053a3f6 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.176173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 468/919] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:45.181571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.182011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.182186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.199590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:45.200773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:45.278161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 452/920] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:45.301302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde6ed0-65d4-4faa-8e29-28cd7053a3f6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.344841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.408133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde6ed0-65d4-4faa-8e29-28cd7053a3f6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.443905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6e346-c2e3-4c72-ac84-ee82d01af7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.444639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6e346-c2e3-4c72-ac84-ee82d01af7bc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.444922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6e346-c2e3-4c72-ac84-ee82d01af7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.445198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6e346-c2e3-4c72-ac84-ee82d01af7bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.456176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6e346-c2e3-4c72-ac84-ee82d01af7bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a1d6e346-c2e3-4c72-ac84-ee82d01af7bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:45.464272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:45.465221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:45.466051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 469/921] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:45.472177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.551455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b6176-9293-437d-9540-c662341d5167 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.552363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b6176-9293-437d-9540-c662341d5167 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.552639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b6176-9293-437d-9540-c662341d5167 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.557630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 470/922] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:45.559066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b5a9e-714b-4484-93d1-20cac24ed4a3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.559351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b5a9e-714b-4484-93d1-20cac24ed4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.559627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b5a9e-714b-4484-93d1-20cac24ed4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.576461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f94b5a9e-714b-4484-93d1-20cac24ed4a3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:45.577591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f94b5a9e-714b-4484-93d1-20cac24ed4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:45.579748 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f3af7-f1f2-46b3-a108-c13b1dad211d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.609655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.621441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.622128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.622408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.622712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.623465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:45.623766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.625135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.634481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 453/923] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:45.639681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.663262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 471/924] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:45.664939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85059-d6a9-4eb9-904b-c71f68de6af4 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.665225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85059-d6a9-4eb9-904b-c71f68de6af4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.665498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85059-d6a9-4eb9-904b-c71f68de6af4 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.667157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 472/925] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:45.671788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.672068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.672342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.695512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:45.697730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:45.722710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.752933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108ea3-4e48-4688-b65e-31d5d1c24ac7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.771236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 473/926] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:45.772153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.796527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.797377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.797660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.797934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.809547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.811132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:45.828823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:45.829680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:45.830552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:45.830552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:45.830552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:45.830552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:45.830552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 454/927] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:45.830864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.873584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.890347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3834e6-7178-4837-b3cf-7ddea5b51685 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.932916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.933249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7ed65-889f-4cd6-81ac-b2dc5bc6f2f6 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:45.933775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.934055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.934372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.935165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.936680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:45.949868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:45.950797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:45.956003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 474/928] 10.222.0.39 () {56 vars in 1132 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:45.956859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:45.971247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:45.972048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:45.972332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:45.972605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:45.973255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:45.975199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:45.980020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:45.988301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:45.989501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:45.990364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 455/929] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:45.991181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.067457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24688aa4-f357-4997-93dc-cb30ae4833c3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.096227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3d6dc-9429-4652-b97f-286b4b945814 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.099091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.108202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.108975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.109261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.109540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.110262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.110567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.112195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.129414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 475/930] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.129414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.135789 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.136456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.136735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.137008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.137715 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.138016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.139441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.144296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 456/931] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.145061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.222731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.243210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70629e16-adb8-4e10-8949-06644f0188ab None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.256019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5707d4-d21d-494c-b821-76dab64f6ba4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.280153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.280820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.281096 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.281366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.282072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.282376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.283773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.288879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 476/932] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.289749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.308693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.309727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.310161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.311108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.321730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.323636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:46.344142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:46.345263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:46.346504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:46.348428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 457/933] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:46.349325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.389610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.391550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71853f-014c-4c56-8ece-c1bb990f49db None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.428193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.428941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.429289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.429650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.430466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.430867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.432295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.437447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 477/934] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.443182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.454563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.470514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d8816-8f09-43b6-ba08-9c8857044938 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.506354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.506974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.507263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.507535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.508239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.508549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.509995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.515438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 458/935] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.516198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.551290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2bc94-3a32-48ee-82a0-9b6469aee590 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.586487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.587175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.587458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.587728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.588560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.588862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.590304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.608464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 478/936] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.611965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.615871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.625719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f89cf-0d09-4b0d-85b7-a24bc062d1a8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.660311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.660995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.661282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.661554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.662544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.663925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.683278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 459/937] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:46.694341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.695666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.734476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb514f9b-7147-4809-866f-453856eabebf None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.780957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.781539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.782196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.782196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.782931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] RBAC: Adding query filter params (interface=public, service_id=be852294fe96473ba089a8e112d5688b, region_id=RegionOne) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:46.783244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.784640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.791167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 479/938] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/endpoints?service_id=be852294fe96473ba089a8e112d5688b&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:46.799713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.854302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a374d6-d3b6-4bb9-bab1-f08433c11f34 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.902300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.903148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.903431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.903704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.921141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.923157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.928889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33edc9d-d207-4618-8dc0-b7d96b27c555 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:46.934170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:46.948429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:46.949344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:46.950212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:46.950212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:46.950212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:46.950212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:46.950212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 460/939] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:46.957525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:46.960799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:46.969951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4e487-05ed-4888-9637-fb5de4dad235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:46.970695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4e487-05ed-4888-9637-fb5de4dad235 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:46.971097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4e487-05ed-4888-9637-fb5de4dad235 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:46.971454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4e487-05ed-4888-9637-fb5de4dad235 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:46.972153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4e487-05ed-4888-9637-fb5de4dad235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:46.974183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4e487-05ed-4888-9637-fb5de4dad235 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:46.981038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 480/940] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:52:47.009608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5abb1c-b8fe-47c6-8240-251de3f16709 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.009922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5abb1c-b8fe-47c6-8240-251de3f16709 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.010172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5abb1c-b8fe-47c6-8240-251de3f16709 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.011841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 481/941] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:47.020127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.020483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.020732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.044590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:47.047223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:47.090286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba378fe-eeb6-46be-8169-5185c551a98e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.135881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.136963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.137316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.137783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.138816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:47.139221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.141897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.154649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 482/942] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:47.160680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 461/943] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:47.162800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.176155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.251328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:47.292507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6430412-d565-41c9-8f3b-fdf2f34aec66 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.305051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04f208-1be1-47ae-9f93-600961af3dfa None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.320822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:47.338771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.339566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.339846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.340117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.346943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.347358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.347985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.349038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.349386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.349651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.350521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.352371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.353918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING keystone.server.flask.common [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:47.363775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:47.365065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:47.366267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:47.370930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 483/944] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:47.371908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.377143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.sql.core [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 20:52:47.377143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 20:52:47.377143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [parameters: {'id': '656e801e31c846a4a435478c98d65230', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 20:52:47.377143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 656e801e31c846a4a435478c98d65230.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 656e801e31c846a4a435478c98d65230. Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:52:47.386121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result.read() Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 13 20:52:47.387147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application yield resource Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise self.value Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application session.commit() Jul 13 20:52:47.388337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.session.flush() Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application compat.raise_( Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 20:52:47.389271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 20:52:47.390169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result.read() Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:52:47.391015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [parameters: {'id': '656e801e31c846a4a435478c98d65230', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:47.392061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:47.392907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 656e801e31c846a4a435478c98d65230. Jul 13 20:52:47.393824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.397025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 462/945] 10.222.0.39 () {60 vars in 1140 bytes} [Wed Jul 13 20:52:47 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:47.408596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.483560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809993a-2a68-4ac5-9470-571db9f12d98 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.520994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.521647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.521925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.522198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.523035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:47.523338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.524905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.537728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 484/946] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:47.544452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.552405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eb3ad-19d0-4dc9-9080-7cc91f3ad175 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.573760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:47.600542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:47.609905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.610740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.611030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.611303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.618465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.620258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:47.634328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:47.635211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:47.636086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:47.640282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 463/947] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:47.641115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:47.670343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7357d01-6d6d-4cce-8624-4f4a6515138d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.706401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9df5-6a38-482c-a8c3-31156d3c82df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.707725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9df5-6a38-482c-a8c3-31156d3c82df None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.708716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9df5-6a38-482c-a8c3-31156d3c82df None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.709188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9df5-6a38-482c-a8c3-31156d3c82df None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/4535c7d4c7a74095b9ccc6777ffee724/roles/82ae094868e64fd49c4f9c9333153a10` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.711751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9df5-6a38-482c-a8c3-31156d3c82df None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=4535c7d4c7a74095b9ccc6777ffee724, role_id=82ae094868e64fd49c4f9c9333153a10)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.713443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9df5-6a38-482c-a8c3-31156d3c82df None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.734812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 485/948] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:52:47 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/4535c7d4c7a74095b9ccc6777ffee724/roles/82ae094868e64fd49c4f9c9333153a10 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:52:47.755772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e00f5-fe78-4fc8-a018-c018a313d727 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:47.796144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:47.796820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:47.797107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:47.797385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:47.798168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:47.798475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:47.800051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411f4d0-2d0f-4f3d-ba01-70b4b2bb357b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:47.823589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 464/949] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:48.823173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5c40-1171-40c4-9f12-6dfbf8b9c283 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:48.823688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5c40-1171-40c4-9f12-6dfbf8b9c283 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:48.823788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e5c40-1171-40c4-9f12-6dfbf8b9c283 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:48.825333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 486/950] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:48.830672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59f60d-bfdf-4901-bfbf-e2219cacaa3e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:48.831078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59f60d-bfdf-4901-bfbf-e2219cacaa3e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:48.831268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59f60d-bfdf-4901-bfbf-e2219cacaa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:48.850261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-bf59f60d-bfdf-4901-bfbf-e2219cacaa3e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:48.851797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bf59f60d-bfdf-4901-bfbf-e2219cacaa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:48.882906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:48.926831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 465/951] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:48.948189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5c40-1171-40c4-9f12-6dfbf8b9c283 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.058693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e5c40-1171-40c4-9f12-6dfbf8b9c283 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.060559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:49.086115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f8d6-f89b-4cb0-ac3a-7f1af437d5f8 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.086399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f8d6-f89b-4cb0-ac3a-7f1af437d5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.086630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4f8d6-f89b-4cb0-ac3a-7f1af437d5f8 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.092118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 466/952] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:49.093374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.093815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.094148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.097708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.099256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.099696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.100211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.101133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.102510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.111877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:49.113300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:49.114329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:49.115366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:49.116932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 487/953] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:49.123521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.198667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 467/954] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:49.221734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.257171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934903d6-a260-416e-9d48-680803cd1366 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.302947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.303755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.304142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.304524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.305344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:49.305755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.307824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.318597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 488/955] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:49.320658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.334335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:49.342586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465082e-2d31-4b28-bc86-5cbfd65be7fe None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.359335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:49.382560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.383887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.384374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.384859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.385738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.387515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:49.401498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:49.402666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:49.403652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:49.406356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 468/956] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:49.413209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.444748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ab118-2a78-4ab2-8900-9bb333f85c7f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.488446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.489927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.490223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.490509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.491280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:49.491593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.493034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.504993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 489/957] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:49.529510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f1aec-ba6a-42a3-ae34-a507bf1648c7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.569696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.570385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.570706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.570988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.571786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:49.572097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.573556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.584462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 469/958] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:49.585941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.690388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:49.698029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837550e-4cca-44b7-bedc-8c01a982ec1a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.732033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.732707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.732993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.733270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.734045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:49.734359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.735814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ad5e4-5668-4a0a-85a1-3c71bcf50127 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.746250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 490/959] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:49.747616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:49.775832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:49.855560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011086a-861f-4e87-9f51-59f097d3d1a0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:49.876889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36133f63-2f53-4448-9b16-3f86f5a52d95 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.877389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36133f63-2f53-4448-9b16-3f86f5a52d95 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.877496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36133f63-2f53-4448-9b16-3f86f5a52d95 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.879080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 491/960] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:49.883927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.884223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.884493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.892405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f3652-ea58-4b26-a83a-145de8d964b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:49.893070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f3652-ea58-4b26-a83a-145de8d964b9 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.893354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f3652-ea58-4b26-a83a-145de8d964b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.893635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f3652-ea58-4b26-a83a-145de8d964b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.894396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f3652-ea58-4b26-a83a-145de8d964b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:49.896346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f3652-ea58-4b26-a83a-145de8d964b9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:49.903213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:49.904492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:49.920274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 470/961] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:49.926993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91641ae-b918-4e18-b6bc-1833b3c0dd4a None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.927305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91641ae-b918-4e18-b6bc-1833b3c0dd4a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.927615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91641ae-b918-4e18-b6bc-1833b3c0dd4a None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.929274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 471/962] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:49.934769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:49.935101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:49.935344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:49.953834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:49.955014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:50.021319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 492/963] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:50.043530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.053450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 472/964] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:50.079574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.097135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:50.146579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:50.169111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e1611-8203-48d2-98ae-ccd6d710eba4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.205968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.206736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.207021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.207298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.223094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:52:50.223572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d7f49-7bd1-4a52-a7a3-8366ae5eb150 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:52:50.234171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:50.235196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:52:50.236061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:50.239188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 493/965] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:50.250083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.265725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.266460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.266785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.267063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.267774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:50.269220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:50.284847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.sql.core [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 20:52:50.284847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 20:52:50.284847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [parameters: {'user_id': '204e91d564bd4a4cb8e180b6fddd0ca1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 20:52:50.284847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80749) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result.read() Jul 13 20:52:50.295661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:52:50.296567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application yield resource Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application next(self.gen) Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise self.value Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application session.commit() Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 20:52:50.297527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.session.flush() Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application compat.raise_( Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 20:52:50.298393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 20:52:50.299347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application conn.query(q) Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application result.read() Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 20:52:50.300189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [parameters: {'user_id': '204e91d564bd4a4cb8e180b6fddd0ca1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:50.301001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 13 20:52:50.304329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 473/966] 10.222.0.39 () {60 vars in 1131 bytes} [Wed Jul 13 20:52:50 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:50.306532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.382733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:50.390475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e82701-faad-4d01-84dc-32e38ab21c5c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.416970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9e7c-94ec-480b-a859-a12d3e8920e2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.428403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.429394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.429894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.430377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.431368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:50.431883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:50.433442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:50.447078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 494/967] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:50.448923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.454603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.455500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.455787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.456117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.459825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:50.465129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:50.467060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:50.483472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:50.484584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 20:52:50.485734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:50.488159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 474/968] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:50.500653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.575888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc7fce-8980-4a3d-8fe0-e316a2ab2368 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.603404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ef613-c8cd-4cc4-a2c1-c459837faea4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.623719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.624645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.626953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.627433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.638178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:50.644476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:50.645975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:50.651318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.652197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.652694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.653130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.654688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:50.655154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:50.657513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:50.665438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:50.666506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:50.667717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:50.667717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:50.667717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:50.667717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:50.667717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 495/969] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:50.675622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.706033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 475/970] 10.222.0.39 () {56 vars in 1101 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:50.754878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:50.804040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1ebdd-52d4-46d6-8aa9-e8c906d9f54f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:50.841629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:50.842467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:50.842865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:50.843271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:50.844113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:50.844517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:50.846010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:50.860637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 496/971] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:50.865819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:50.972318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51905699-dd97-4ced-a8d2-2f3488983004 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:51.007984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:51.008732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.009014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.009286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.016594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:51.018107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:52:51.033308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:51.034217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:51.035214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 476/972] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:52:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:51.042250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:51.146434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c57e15-c33c-48ab-9b51-5a762e215b72 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:51.164461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:51.181616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:51.182288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.182571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.182873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.183717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:51.184025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:51.185452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:51.206685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 497/973] 10.222.0.39 () {56 vars in 1149 bytes} [Wed Jul 13 20:52:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:51.208744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:51.260800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:51.321631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ea2d2-8c74-4ccc-9a09-07af1dcbf7ea None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:51.355000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0a86-2005-41a6-9da3-81b7355d360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:51.355641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0a86-2005-41a6-9da3-81b7355d360d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.355921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0a86-2005-41a6-9da3-81b7355d360d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.356194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0a86-2005-41a6-9da3-81b7355d360d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.421356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf0a86-2005-41a6-9da3-81b7355d360d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:51.422823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf0a86-2005-41a6-9da3-81b7355d360d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:51.469928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:51.525441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 477/974] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:52:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:52:51.526777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:51.625591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6954b-1922-4e9b-afb5-5880ab320637 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:51.636120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e8ef-b73f-47fd-b57a-37069577b103 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.636120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e8ef-b73f-47fd-b57a-37069577b103 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.636120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780e8ef-b73f-47fd-b57a-37069577b103 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.637513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 478/975] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:51.642495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.642817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.643096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.661940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:51.663191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:51.663752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:51.664010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.664290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.664560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.674965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:51.676397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:52:51.695103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:51.696006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:51.697568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:52:51.697568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:52:51.697568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:52:51.697568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:51.697568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 498/976] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:51.701729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:51.732277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:51.742668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 479/977] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:51.762449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:51.814190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff32f-1605-4d57-8985-8af68545ff24 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:51.844074 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:51.851208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:51.851877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.852602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.852900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.853639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:51.853953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:51.855351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:51.866541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b700957-8d91-463e-9630-8ad1fd3e2373 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:51.871809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 499/978] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:51.872629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:51.907760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522375a-2fac-4549-a0cc-dee39c5b6dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:51.908726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522375a-2fac-4549-a0cc-dee39c5b6dac None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:51.909010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522375a-2fac-4549-a0cc-dee39c5b6dac None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:51.909285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522375a-2fac-4549-a0cc-dee39c5b6dac None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:51.910001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522375a-2fac-4549-a0cc-dee39c5b6dac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:51.911410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522375a-2fac-4549-a0cc-dee39c5b6dac None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:51.934740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 480/979] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:51 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:52:51.967298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:51.976947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7e7c4-a0d9-46f7-a6a7-6c731eac8844 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.021696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.022505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.022814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.023086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.027602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d3c14-8ac5-4d90-a002-7bc7d7ef942d None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.027870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d3c14-8ac5-4d90-a002-7bc7d7ef942d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.028086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d3c14-8ac5-4d90-a002-7bc7d7ef942d None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.029638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 481/980] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:52.030923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.032394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.035053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.035348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.035678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:52:52.048283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:52.049616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:52.051949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 500/981] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:52.054859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:52.056137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.056292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:52.134895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 482/982] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:52.157255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.178315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9561008-ecd9-48a7-bb3a-cb18eb797153 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.218697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.219356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.219638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.219918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.220703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:52.221116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.222535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.232133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:52.235093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 501/983] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:52.241227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.245740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:52.277844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c014da-7932-4340-8e52-87fc41fb401b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.320643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.322204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.322726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.323221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.323987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.325382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:52.337978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:52.339089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:52.339933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 483/984] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:52.345013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.360647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aef419-92be-4433-9bcf-b75c011c4a17 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.399270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.399969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.400258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.400647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.401483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] RBAC: Adding query filter params (role.id=82ae094868e64fd49c4f9c9333153a10, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=4535c7d4c7a74095b9ccc6777ffee724) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:52.401797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.403371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.413510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 502/985] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/role_assignments?user.id=4535c7d4c7a74095b9ccc6777ffee724&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=82ae094868e64fd49c4f9c9333153a10 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:52.451506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88e913-3f99-4a42-a7e2-3f6f0f6fa749 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.488540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.489797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.490082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.490359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.491119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:52.491430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.492826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.494222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:52.504685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 484/986] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:52.505423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.579340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:52.619880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e26e0-5e14-4ace-876d-1d156f6f2c5b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.653427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.654020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.654308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.654588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.655327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:52.655635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.656995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.662821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 503/987] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:52.668361 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.769822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900268f-fade-4815-b666-c54e3f37b429 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.803881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.804539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.804822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.805100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.805899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] RBAC: Adding query filter params (interface=admin, service_id=be852294fe96473ba089a8e112d5688b, region_id=RegionOne) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:52.806211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.807621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ddcb8c-1d9f-49cf-b921-489cf8daae37 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.817359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 485/988] 10.222.0.39 () {56 vars in 1249 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/endpoints?service_id=be852294fe96473ba089a8e112d5688b&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:52.818326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:52.920767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192dd120-5b24-4f86-afd7-b8dfa8f169c9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:52.950306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:52.956079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4565091-4325-4601-89eb-8f698a134e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:52.957203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565091-4325-4601-89eb-8f698a134e01 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:52.957551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565091-4325-4601-89eb-8f698a134e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:52.958369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4565091-4325-4601-89eb-8f698a134e01 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:52.959099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565091-4325-4601-89eb-8f698a134e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:52.960469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4565091-4325-4601-89eb-8f698a134e01 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:52.967159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 504/989] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:52 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:52:53.965892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ab030-5937-418c-ac69-2ab9b0dbca55 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:53.966415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ab030-5937-418c-ac69-2ab9b0dbca55 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:53.966493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ab030-5937-418c-ac69-2ab9b0dbca55 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:53.968010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 486/990] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:53.972870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:53.973759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:53.973945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:53.991151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:53.992323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:54.063147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 505/991] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:54.085875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ab030-5937-418c-ac69-2ab9b0dbca55 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.148683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:54.189888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ab030-5937-418c-ac69-2ab9b0dbca55 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.222985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.223738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.224016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.224290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.225425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.226827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:54.238388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:54.239488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:54.240410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:54.242771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 487/992] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:54.245550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.350155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3738d-fdc3-4a0b-b1cf-13f5e6275646 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.375238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:54.384621 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.385292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.385573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.385846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.386549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:54.386882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.388242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d6fc5-6191-411c-8a2b-8cc5e032b0b7 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.393192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 506/993] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:54.399070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.475099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4d777-8ecf-4280-8313-0862099e70fd None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.475409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4d777-8ecf-4280-8313-0862099e70fd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.475592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4d777-8ecf-4280-8313-0862099e70fd None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.476953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 507/994] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:54.481846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.482131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.482468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.499262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:54.500452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:54.502902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383e8a2-e25c-4577-aab8-1e446b99d7a5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.533155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:54.543977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.544652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.544937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.545217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.545930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:54.546243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.547656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.553042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 488/995] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:54.558683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.578182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 508/996] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:54.600645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.668871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000497bf-dac7-4107-8b06-7119d6874c24 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.704541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb916e-c181-4f95-a15e-fbbc1ded1f6d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.705299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.706048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.706241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.706514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.707343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] RBAC: Adding query filter params (interface=public, service_id=2ea6dffec4ea4ca79cb8278b491d3bb9, region_id=RegionOne) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:54.708173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.709533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.714566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 489/997] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/endpoints?service_id=2ea6dffec4ea4ca79cb8278b491d3bb9&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:54.719920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.730825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:54.746419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.747237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.747522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.747795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] PATH_INFO: `/v3/services/image` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.748512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.749875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:54.761872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:54.762894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:52:54.763708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 509/998] 10.222.0.39 () {56 vars in 1105 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:52:54.768873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.782341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:54.831423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e4a71-4302-4505-8677-e228840a9d26 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.867373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.868041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.868393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.868598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.869275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.870664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.876061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ebbfc-8196-4bfb-af2b-bc30cdcbce44 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:54.877339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 490/999] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:52:54.915193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:54.915879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.916172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.916450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.917177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] RBAC: Adding query filter params (name=image) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:54.917487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:54.918901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e709014-a2ad-4887-a75e-4a0a98d99d27 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:54.924839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 510/1000] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:54.930992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:54.995956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7addc05-23cf-4cf9-93b9-daca85e36acf None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:54.996471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7addc05-23cf-4cf9-93b9-daca85e36acf None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:54.996471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7addc05-23cf-4cf9-93b9-daca85e36acf None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:54.998227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 511/1001] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:55.003977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:55.004834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:55.005112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:55.022560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:55.023822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:55.047592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324115b-86d8-46f1-b458-33b670a137b2 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:55.069082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:55.073605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:55.088973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:55.089740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:55.090097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:55.090453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:55.091275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] RBAC: Adding query filter params (type=image) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:55.091590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:55.092965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:55.097703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 491/1002] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:55.109412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 512/1003] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:55.133289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:55.243428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6fccb-26e9-45ba-8f0c-62e954bb28dd None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:55.283179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:55.284001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:55.284286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:55.284566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:55.285228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:55.287424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:55.299558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:55.300557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:55.302036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 492/1004] 10.222.0.39 () {56 vars in 1126 bytes} [Wed Jul 13 20:52:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:55.307312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:55.416554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a72cb0-389f-48f9-93d2-f8f32a8a20c6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:55.450679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:55.451349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:55.451635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:55.451914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:55.452626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:55.452936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:55.454331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:55.459466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 513/1005] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:55.464925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:55.514335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:55.572436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6bc61-a378-411e-8bce-2bca574bbae7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:55.607713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:55.608387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:55.608674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:55.608955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:55.609665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:55.609975 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:55.611378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2749c-c42e-44f9-b38f-366793e7967d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:55.616636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 493/1006] 10.222.0.39 () {56 vars in 1122 bytes} [Wed Jul 13 20:52:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:55.622308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:55.669098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:55.726874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75f19f-bf31-4d4e-a932-4c2acbcd41ff None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:55.760425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c7817-3f93-4868-a7f2-2a2814db9147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:55.761082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7817-3f93-4868-a7f2-2a2814db9147 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:55.761363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7817-3f93-4868-a7f2-2a2814db9147 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:55.761640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c7817-3f93-4868-a7f2-2a2814db9147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:55.762388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7817-3f93-4868-a7f2-2a2814db9147 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:55.763785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c7817-3f93-4868-a7f2-2a2814db9147 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:55.783741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 514/1007] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:56.924836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22be98-aa57-4c0c-bde9-9100f5f65cd0 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:56.925375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22be98-aa57-4c0c-bde9-9100f5f65cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:56.925375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22be98-aa57-4c0c-bde9-9100f5f65cd0 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:56.930360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 494/1008] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:56.933463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1626177-747e-4982-9d94-93c5b347fcbe None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:56.933975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1626177-747e-4982-9d94-93c5b347fcbe None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:56.934056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1626177-747e-4982-9d94-93c5b347fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:56.950838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a1626177-747e-4982-9d94-93c5b347fcbe None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:56.952009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a1626177-747e-4982-9d94-93c5b347fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:57.026310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 515/1009] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:57.048963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22be98-aa57-4c0c-bde9-9100f5f65cd0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:57.150862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22be98-aa57-4c0c-bde9-9100f5f65cd0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:57.162129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:57.171576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc5ed-7c95-4124-ada0-c9b075eb0e55 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.171871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc5ed-7c95-4124-ada0-c9b075eb0e55 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.172152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc5ed-7c95-4124-ada0-c9b075eb0e55 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.173784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 516/1010] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:57.178330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.178568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.178896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.187078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:57.187876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.188155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.188425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.189072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:57.190420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:57.195211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:57.196395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:57.201936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:57.202948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 20:52:57.203792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:57.207032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 495/1011] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:52:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:57.209373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:57.221556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:57.271324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 517/1012] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:57.294223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:57.329465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cb908-fbe6-4fc0-9dde-11c8cbd97cc4 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:57.365915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:57.366585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.366898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.367184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.367891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:57.368195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:57.369560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:57.374606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 496/1013] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:57.380237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:57.411750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab3548-9438-4fda-a3f9-7b433f898eb3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:57.420739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:57.450792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:57.451502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.451787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.452065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.452789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:57.454160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:57.474653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 518/1014] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:57 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:52:57.486662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bdff4-a2ea-469c-b721-98c409996e40 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:57.532503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:57.533178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.533463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.533737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.534483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:57.534821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:57.536245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b6e63-d406-4633-9877-f67ea45c90bc None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:57.542433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 497/1015] 10.222.0.39 () {56 vars in 1112 bytes} [Wed Jul 13 20:52:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:52:57.548050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:57.569624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:57.655916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452b3b0-f526-49cc-bb4f-dbb1cff64fad None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:57.693972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa99bad-b02c-463e-95de-191f5c280331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:57.694638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa99bad-b02c-463e-95de-191f5c280331 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.694937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa99bad-b02c-463e-95de-191f5c280331 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.695211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa99bad-b02c-463e-95de-191f5c280331 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.695957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa99bad-b02c-463e-95de-191f5c280331 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:57.697431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa99bad-b02c-463e-95de-191f5c280331 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:57.716825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 519/1016] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:52:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:57.861906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ccfdf-9a3e-4a4d-83cf-1449aa42e90e None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.862245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ccfdf-9a3e-4a4d-83cf-1449aa42e90e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.862479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ccfdf-9a3e-4a4d-83cf-1449aa42e90e None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.864629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 498/1017] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:57.869774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:57.870069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:57.870351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:57.888219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:57.889611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:57.961058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:57.971043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 520/1018] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:57.992466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ccfdf-9a3e-4a4d-83cf-1449aa42e90e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:58.093358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:58.099408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ccfdf-9a3e-4a4d-83cf-1449aa42e90e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:58.133994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc6de3-ffe6-463f-8421-9675635a535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:58.134824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc6de3-ffe6-463f-8421-9675635a535f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:58.135110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc6de3-ffe6-463f-8421-9675635a535f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:58.135388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc6de3-ffe6-463f-8421-9675635a535f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:58.136622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc6de3-ffe6-463f-8421-9675635a535f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:58.138110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc6de3-ffe6-463f-8421-9675635a535f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:58.144524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 499/1019] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:58.146077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:58.248387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d0e52-3e9f-42f2-b11f-b97b4fd20efd None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:58.287950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e108a-fbcc-43fa-bcb6-3cff98137c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:58.288622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e108a-fbcc-43fa-bcb6-3cff98137c98 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:58.288908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e108a-fbcc-43fa-bcb6-3cff98137c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:58.289188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e108a-fbcc-43fa-bcb6-3cff98137c98 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:58.289913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e108a-fbcc-43fa-bcb6-3cff98137c98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:58.291408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e108a-fbcc-43fa-bcb6-3cff98137c98 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:58.322042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 521/1020] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:52:58 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:52:59.535170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7f40-f856-49b0-b19d-5009b8287d54 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.535662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7f40-f856-49b0-b19d-5009b8287d54 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.535662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7f40-f856-49b0-b19d-5009b8287d54 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.537352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 500/1021] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:52:59.542333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.542759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.542918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.559766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:59.560939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:59.606861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:59.628216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 522/1022] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:52:59.650848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7f40-f856-49b0-b19d-5009b8287d54 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:59.753620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7f40-f856-49b0-b19d-5009b8287d54 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:59.787612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:59.788380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.788682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.788963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] PATH_INFO: `/v3/services/image` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.789616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:59.791009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-23f9d9a4-76da-499d-adeb-5ba22f12e821 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:52:59.803216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:52:59.804182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:52:59.805027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:52:59.806411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 501/1023] 10.222.0.39 () {56 vars in 1105 bytes} [Wed Jul 13 20:52:59 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:52:59.810272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:59.825067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b7b76-004a-4815-8c9a-8c736e9aca44 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.825345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b7b76-004a-4815-8c9a-8c736e9aca44 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.825626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b7b76-004a-4815-8c9a-8c736e9aca44 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.827158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 502/1024] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:52:59.831671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.831960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.832281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.850382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:52:59.851584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:52:59.876338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:52:59.912034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7083f-020a-4dc2-b003-444d0e9f517b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:52:59.923652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 503/1025] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:52:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:52:59.947407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:52:59.947861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:52:59.948541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:52:59.948825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:52:59.949101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:52:59.950094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] RBAC: Adding query filter params (name=image) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:52:59.951014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:52:59.952597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:52:59.958342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 523/1026] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:52:59.963520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.013638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.055591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb42ac6-974f-4466-8b47-cac6aa1c8a87 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.071802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac85779-42f3-4d12-95da-c3d6872c5572 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.090911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.091726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.092011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.092289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.103962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:00.107055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.107716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.108029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.108309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.109017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] RBAC: Adding query filter params (type=image) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:00.109324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.110707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:00.112154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:00.113081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:53:00.113951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:00.118037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 524/1027] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:52:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:00.119107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.122949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 504/1028] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Jul 13 20:52:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:00.123817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.147570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.229114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722f98b-2a74-4e20-8a0f-8c5aae4cde70 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.229497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58465be9-d01d-49f2-a938-3763d63c17a9 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.241650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.265072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.265735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.266023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.266303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.267127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] RBAC: Adding query filter params (interface=public, service_id=2cfc8e227f3c42b480883734946ab313, region_id=RegionOne) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:00.267443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.268822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.281127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.281237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 505/1029] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/endpoints?service_id=2cfc8e227f3c42b480883734946ab313&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:00.281313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.281313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.281313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.281313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:00.281313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.281313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.281887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.293510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 525/1030] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:00.294504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.373364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.390417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b699d65-e863-443f-be6c-577f0965ff21 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.400609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f115d71-3cac-49f1-b2d9-075aceee3efd None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.427747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46ec59-d691-422f-bb74-15d141c72630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.428413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ec59-d691-422f-bb74-15d141c72630 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.428698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ec59-d691-422f-bb74-15d141c72630 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.428971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ec59-d691-422f-bb74-15d141c72630 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.429662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46ec59-d691-422f-bb74-15d141c72630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.431054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46ec59-d691-422f-bb74-15d141c72630 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.434558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.435380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.435660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.435929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.437762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 506/1031] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 20:53:00.438905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b512d-444a-41aa-b333-eae08254d686 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.439196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b512d-444a-41aa-b333-eae08254d686 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.439624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b512d-444a-41aa-b333-eae08254d686 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.440857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 507/1032] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:00.446900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.447602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.447602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.449690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.451307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.468224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:00.469729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:00.470580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:00.471637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:00.472807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:00.472807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:00.472807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:00.472807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:00.472807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 526/1033] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:00.477813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.507385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.547974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 508/1034] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:00.573026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.596839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5260a-ff3f-4839-b56e-a17321309b01 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.636455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.637480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.638022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.638498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.639401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:00.639888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.641253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.659473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 527/1035] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:00.661225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.661681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.687737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc449-aaf5-4e88-b5e4-7092560aebfd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.724319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.725111 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.725390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.725823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.726847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.728266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.730041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 509/1036] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:00.735401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.750515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:00.770176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3d49-b3d6-4961-8515-f507951712ae None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.806381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.807593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.807593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.807769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.815675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.818056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:53:00.833675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:00.834727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:00.835680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 528/1037] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:00.840468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:00.843895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cbc22-657a-4546-93ab-8b92737501b9 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.880795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.881525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.881808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.882132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.882880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:00.884306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:00.918733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 510/1038] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:53:00 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:00.959575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaebe90-6303-4b23-ad8a-140790ec5eb0 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:00.996772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:00.997450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:00.997733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:00.998007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:00.998906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:00.999192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.000629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.020113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 529/1039] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:53:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:01.026058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.029415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:01.137062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76a782-b705-4638-8eb7-facf641ec8cb None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.171821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.172395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.172677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.172953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.239955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.241502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.243374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:01.351046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 511/1040] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:01.356478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.359414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:01.458728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd0f80-9ecf-475c-aca5-4cefacd9233a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.492587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.493376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.493659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.493931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.503966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.505410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:01.523098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:01.523979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:01.529833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:01.529833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:01.529833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:01.529833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:01.529833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 530/1041] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:01.531385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.631850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400143-2dae-4dea-8ae8-06b5996ab10d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.633366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:01.667844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.668513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.668796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.669073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.669794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:01.670098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.671563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.687579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 512/1042] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:01.688875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.747875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:01.793317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646773a3-8b41-420e-a924-aec2d1055cdb None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.825582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.826360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.826667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.826949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.834074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.835535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:53:01.849467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:01.850336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:53:01.851602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:01.854920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 531/1043] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:01.856390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:01.958470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc4cfb-3f9a-4f46-87a1-8ae465ba9319 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:01.991483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:01.992447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:01.992927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:01.993423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:01.994416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:01.994950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:01.996796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559dd21-9b35-4db3-a632-161e294ad976 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.012308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 513/1044] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:53:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:02.017858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.121643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff1097-cdb0-4e97-9c6d-865353188b84 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.135646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:02.156866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.157516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.157794 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.158068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.158925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] RBAC: Adding query filter params (role.id=82ae094868e64fd49c4f9c9333153a10, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a2048d28ca3b42d8b8d82f4754e44433) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:02.159233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.160689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314f8c-0143-409c-912d-21fa5a70003c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.166536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 532/1045] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:02 2022] GET /identity/v3/role_assignments?user.id=a2048d28ca3b42d8b8d82f4754e44433&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=82ae094868e64fd49c4f9c9333153a10 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:02.461829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a57c8-9b62-4a0b-8a5b-9d5effc7b43b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.462195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a57c8-9b62-4a0b-8a5b-9d5effc7b43b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.462429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a57c8-9b62-4a0b-8a5b-9d5effc7b43b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.463926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 514/1046] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:02.468995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.469289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.469568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.486190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:02.487405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:02.554461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 533/1047] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:02.576935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a57c8-9b62-4a0b-8a5b-9d5effc7b43b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.601933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:02.683324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a57c8-9b62-4a0b-8a5b-9d5effc7b43b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.721470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.723132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.723558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.723971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] PATH_INFO: `/v3/services/image` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.724911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.726961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:02.739040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:02.740048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 20:53:02.740932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:02.742943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 515/1048] 10.222.0.39 () {56 vars in 1105 bytes} [Wed Jul 13 20:53:02 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:02.746481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.844131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2533e7-a583-41f1-bbd4-d718724ae32b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:02.853734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:02.879647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:02.880318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:02.880602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:02.880878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:02.881584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] RBAC: Adding query filter params (name=image) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:02.881891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:02.883287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0a221-a764-45ab-8ea5-21a061bfeffa None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:02.892932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 534/1049] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:02.894189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:02.980971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:02.997653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f2d94-d9af-47b3-8686-81a52c30b929 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.018189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a6ae9-b2cf-4bb1-a72c-b8930081de30 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.018474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a6ae9-b2cf-4bb1-a72c-b8930081de30 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.018788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a6ae9-b2cf-4bb1-a72c-b8930081de30 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.020819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 535/1050] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:03.025373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.025657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.025927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.033728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.034396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.034712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.034992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.036246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] RBAC: Adding query filter params (type=image) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:03.036558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.038607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.042849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:03.048156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 516/1051] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:03.048304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:03.048916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.113689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 536/1052] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:03.136193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.149531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24778a-9225-4653-9947-9b712de0a590 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.186510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.187539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.187827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.188104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.188848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.190210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.213416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 517/1053] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:53:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:03.247103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e24ef7-a078-4154-ad9a-dcb6ae872b6e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.249369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:03.303456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.304264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.304547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.304821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.316651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:03.324363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:03.325386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:03.326218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:03.327554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 537/1054] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:03.332473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.342295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:03.441197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ec2d6-924b-4e30-8f0d-34960268fd90 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.477562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.478236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.478520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.478827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.479577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:03.479882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.481249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.489688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 518/1055] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:03.495247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.598773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d5f850-f42f-4ac2-ba52-c7fae22583ec None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.632786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.633802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.634193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.634574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] PATH_INFO: `/v3/users/e0c6c744c81a493fb8f64f714dc58408` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.649378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] RBAC: Authorizing `identity:get_user(user_id=e0c6c744c81a493fb8f64f714dc58408)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.650933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.657285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 538/1056] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/users/e0c6c744c81a493fb8f64f714dc58408 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:03.658476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.727144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:03.764384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c693eac-f02c-4021-bab7-d3f36a3dcb58 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:03.798351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:03.799046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:03.799335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:03.799616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:03.866884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:03.868345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:03.932595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:03.968857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 519/1057] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:53:03.974488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:03.993099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:04.076222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36274386-7a16-4dad-9a2f-a89236486205 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:04.109664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287de4d4-b4d9-44e0-80b7-04b10e016c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:04.110519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287de4d4-b4d9-44e0-80b7-04b10e016c82 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.110829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287de4d4-b4d9-44e0-80b7-04b10e016c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.111109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287de4d4-b4d9-44e0-80b7-04b10e016c82 None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.112287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287de4d4-b4d9-44e0-80b7-04b10e016c82 None admin] RBAC: Authorizing `identity:get_project(project_id=643b223c5a7b4f48a0fa7b0278745388)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:04.114253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287de4d4-b4d9-44e0-80b7-04b10e016c82 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:04.116236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 539/1058] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:03 2022] GET /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:04.122068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:04.153091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6824753-e73c-4a3b-9f95-afc619ad1d32 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.153365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6824753-e73c-4a3b-9f95-afc619ad1d32 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.153641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6824753-e73c-4a3b-9f95-afc619ad1d32 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.155283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 540/1059] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:04.160540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.160822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.161095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.182822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:04.184021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:04.227753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ee1a-8904-41da-ae0d-0e45f519d658 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:04.253221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 541/1060] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:04.268143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:04.268143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.270732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.273945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.273945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=643b223c5a7b4f48a0fa7b0278745388, user.id=e0c6c744c81a493fb8f64f714dc58408) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:04.273945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:04.274171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:04.276687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:04.281406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 520/1061] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:04 2022] GET /identity/v3/role_assignments?user.id=e0c6c744c81a493fb8f64f714dc58408&scope.project.id=643b223c5a7b4f48a0fa7b0278745388&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:04.293437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:04.392497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e7088-1cff-4cba-a44e-765599f3f2a1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:04.430675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:04.431504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.431788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.432063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.443239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:04.451413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:04.452325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:04.453154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 542/1062] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Jul 13 20:53:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:04.458743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:04.490283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:04.559127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850b2a6-3ddc-4b41-85c7-655f485bc241 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:04.592088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:04.592750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.593035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.593313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.594061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:04.594371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:04.595757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:04.604850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 521/1063] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:53:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:04.609552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:04.686653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:04.712945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da8547-5d74-4d80-814b-09b5b48e4cda None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:04.746860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:04.747645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.747925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.748193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.759256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:04.760691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:04.777394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:04.778290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:04.783667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:04.783667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:04.783667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:04.783667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:04.783667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 543/1064] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:04.785925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:04.878744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:04.887285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716d731-4aae-4c3a-9a65-285fead1347b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:04.920779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:04.921449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:04.921732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:04.922004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:04.922746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:04.923050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:04.924398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:04.941317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 522/1065] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:04.942808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.041976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13137b-bffe-4878-a8c5-68d26904a389 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.067061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:05.076718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.077497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.077778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.078051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.086087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.087533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:53:05.101553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:05.102458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:05.103430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 544/1066] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:53:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:05.109707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.208080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab02b63-9093-4630-9ce9-c1b0eb98d978 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.241575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.242242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.242522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.242821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.243654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:05.243957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.245365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6868f5e-4409-4d37-8ee5-ecb2def7861e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.266652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 523/1067] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:05.270487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.275404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de946b-e397-4e5c-838e-7cff55adfac4 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.275758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de946b-e397-4e5c-838e-7cff55adfac4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.275957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de946b-e397-4e5c-838e-7cff55adfac4 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.277466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 524/1068] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:05.282859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832059-b709-4c26-808c-96d664f23d86 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.283160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832059-b709-4c26-808c-96d664f23d86 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.283444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88832059-b709-4c26-808c-96d664f23d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.301745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-88832059-b709-4c26-808c-96d664f23d86 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:05.302949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-88832059-b709-4c26-808c-96d664f23d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:05.310928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:05.376386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 525/1069] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:05.377832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db1468-6165-4bfc-b73d-b12497ac6447 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.399188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832059-b709-4c26-808c-96d664f23d86 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.414975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.415771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.416061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.416548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.427581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.428933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:05.447503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:05.448407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:05.449301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:05.449301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:05.449301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:05.449301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:05.449301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 545/1070] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:05.454174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.497799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:05.505412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88832059-b709-4c26-808c-96d664f23d86 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.539968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.540616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.540901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.541173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.553238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:05.567711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40137-3f28-4b08-a426-1dff6131fcea None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.603479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.604134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.604420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.604861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.605422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:05.605735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.607135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.611773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.613711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.620185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 546/1071] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:05.625835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.722251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 526/1072] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:53:05.728577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.730158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:05.734177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a6b13-8bcf-4c46-9f47-515727b46b91 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.772683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:05.773888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.775003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.775278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.775784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.783954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.787059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:53:05.804969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:05.805895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:05.812207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 547/1073] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:05.812715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.830980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b051ced-e767-4ff0-9b27-e05c4b1da061 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.866862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.867654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.867936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.868209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.879590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.882143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:05.900809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:05.901828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:05.903055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:05.903055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:05.903055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:05.903055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:05.908286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 527/1074] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:05.909003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:05.919239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbd73a-2d54-4fc1-83b2-a1b7195941a1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:05.956292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:05.956961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:05.957249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:05.957530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:05.958324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:05.958652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:05.960089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:05.973618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 548/1075] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:05.979442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:06.000081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:06.014293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:06.023233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317cc1f-25a7-418c-ac74-f3541aaa363a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:06.063446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:06.064117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.064712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.064992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.065873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:06.066212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:06.067603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2ab50-8fca-45df-944a-d5f0e4265912 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:06.079252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 528/1076] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:06.092206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696b110-f61f-4982-9c30-98f25cfa870c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:06.130222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f34f6-6017-4a5a-ac41-5061e5b31935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:06.131241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f34f6-6017-4a5a-ac41-5061e5b31935 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.131522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f34f6-6017-4a5a-ac41-5061e5b31935 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.131792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f34f6-6017-4a5a-ac41-5061e5b31935 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a2048d28ca3b42d8b8d82f4754e44433/roles/82ae094868e64fd49c4f9c9333153a10` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.134167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f34f6-6017-4a5a-ac41-5061e5b31935 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=a2048d28ca3b42d8b8d82f4754e44433, role_id=82ae094868e64fd49c4f9c9333153a10)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:06.135670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f34f6-6017-4a5a-ac41-5061e5b31935 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:06.152526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 549/1077] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:53:05 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/a2048d28ca3b42d8b8d82f4754e44433/roles/82ae094868e64fd49c4f9c9333153a10 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:53:06.309706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeadc9-d353-46c9-bc9f-b4b517f72f63 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.310006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeadc9-d353-46c9-bc9f-b4b517f72f63 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.310292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeadc9-d353-46c9-bc9f-b4b517f72f63 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.311873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 529/1078] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:06.317233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.317642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.318028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.336081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:06.337273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:06.413459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 550/1079] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:06.435478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeeadc9-d353-46c9-bc9f-b4b517f72f63 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:06.536342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeeadc9-d353-46c9-bc9f-b4b517f72f63 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:06.569309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:06.570099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.570384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.570685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.580609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:06.588861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:06.589941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:06.590961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 530/1080] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:06.596122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:06.606187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:06.693675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31db727-ba48-4d95-a813-b23e5ec22f5f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:06.730394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:06.731084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.731365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.731638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.732374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:06.732677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:06.734026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:06.745444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 551/1081] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:06.751195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:06.767581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:06.850391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09309-55f0-42ad-ad24-36f71c763edf None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:06.883625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:06.884490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:06.884775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:06.885055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] PATH_INFO: `/v3/users/e0c6c744c81a493fb8f64f714dc58408` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:06.886234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] RBAC: Authorizing `identity:get_user(user_id=e0c6c744c81a493fb8f64f714dc58408)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:06.887699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:06.889345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 531/1082] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:06 2022] GET /identity/v3/users/e0c6c744c81a493fb8f64f714dc58408 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:06.895317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:06.961980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:06.992373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81904112-f4ee-47a2-b530-640999cd8d7c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:07.025226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b8109-ddbe-4a0e-8ca0-26d76fcda9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:07.026065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b8109-ddbe-4a0e-8ca0-26d76fcda9dd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:07.026343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b8109-ddbe-4a0e-8ca0-26d76fcda9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:07.026641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b8109-ddbe-4a0e-8ca0-26d76fcda9dd None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:07.027794 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b8109-ddbe-4a0e-8ca0-26d76fcda9dd None admin] RBAC: Authorizing `identity:get_project(project_id=643b223c5a7b4f48a0fa7b0278745388)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:07.029211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b8109-ddbe-4a0e-8ca0-26d76fcda9dd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:07.030939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 552/1083] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:06 2022] GET /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:07.037050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:07.115535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:07.134553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adba27-3acc-4387-abaf-63f7b3cb98c3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:07.167743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bda94-046e-4601-a83d-e91d2028ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:07.168634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bda94-046e-4601-a83d-e91d2028ebc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:07.168919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bda94-046e-4601-a83d-e91d2028ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:07.169196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bda94-046e-4601-a83d-e91d2028ebc9 None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388/users/e0c6c744c81a493fb8f64f714dc58408/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:07.171177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bda94-046e-4601-a83d-e91d2028ebc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=643b223c5a7b4f48a0fa7b0278745388, user_id=e0c6c744c81a493fb8f64f714dc58408, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:07.172610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bda94-046e-4601-a83d-e91d2028ebc9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:07.185641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 532/1084] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:53:07 2022] PUT /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388/users/e0c6c744c81a493fb8f64f714dc58408/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:08.172800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b223-91d0-4598-a3c8-7525e3e6772a None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:08.173355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b223-91d0-4598-a3c8-7525e3e6772a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:08.173355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b223-91d0-4598-a3c8-7525e3e6772a None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:08.174737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 553/1085] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:08.179937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:08.180342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:08.180606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:08.197392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:08.198566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:08.264236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 533/1086] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:08.287892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b223-91d0-4598-a3c8-7525e3e6772a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:08.386854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b223-91d0-4598-a3c8-7525e3e6772a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:08.420067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:08.420868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:08.421151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:08.421426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:08.428689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:08.432807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:08.441900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:08.443046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 20:53:08.444079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:08.446763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 554/1087] 10.222.0.39 () {56 vars in 1120 bytes} [Wed Jul 13 20:53:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:08.449116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:08.516306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:08.544832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8c129-4ce6-4446-a17d-c0594d1532ae None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:08.577528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:08.578185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:08.578466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:08.578778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:08.579520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:08.579823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:08.581183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:08.588874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 534/1088] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:53:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:08.594581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:08.690930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec9c25-8c97-4bcb-8be6-cc86a0dd0613 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:08.724827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:08.725609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:08.725893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:08.726168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:08.736007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:08.737436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:08.754981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:08.756021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:08.756945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:08.756945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:08.756945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:08.756945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:08.756945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 555/1089] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:08.761856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:08.858039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42645c-dba5-49f7-a653-b599433b99c0 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:08.890310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:08.890990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:08.891273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:08.891547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:08.892257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:08.892560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:08.893910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:08.895394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:08.906293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 535/1090] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:08.912413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:08.960511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.010171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd56b4-028e-4b07-a597-ecdbf232ea07 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.044311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.045095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.045380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.045655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.052675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.054110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-8f38cf3c-954a-4c38-a21b-ea7002afe25f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:53:09.068151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:09.069070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:09.070188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 556/1091] 10.222.0.39 () {56 vars in 1093 bytes} [Wed Jul 13 20:53:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:09.075050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.170727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0053d-75a1-4ed6-8882-1d1c5d957e13 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.172490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed39267-f1e5-4ac0-bc09-d84d33b2811e None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.172781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed39267-f1e5-4ac0-bc09-d84d33b2811e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.173061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed39267-f1e5-4ac0-bc09-d84d33b2811e None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.174428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 557/1092] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:53:09.179259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.179540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.179816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.195933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:09.197113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:09.204349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.205020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.205305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.205582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.206423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:09.207663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.209086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.226418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 536/1093] 10.222.0.39 () {56 vars in 1133 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:09.232138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.264893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 558/1094] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:09.284679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.289493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.328331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.331380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716782a-8994-4586-b392-b4951d18fad8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.365392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.366038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.366324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.366601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.385419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483eb85-385b-45c9-9712-64555e9393b4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.419523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.420299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.420580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.420852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.430696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:09.431848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.433279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:09.438304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:09.439427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:09.440684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 559/1095] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:09.445944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.485968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.532734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 537/1096] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:53:09.538662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.541536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.544303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d84b8-ea00-4c8c-a736-a2bf53f6d2b8 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.578288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.578990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.579271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.579547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.580295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:09.580613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.582557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.590902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 560/1097] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:09.596461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.636575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b607f0c-3789-4cb9-b08d-cc3844bfb342 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.669728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45508641-f686-4acd-94de-c8916e829bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.670571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45508641-f686-4acd-94de-c8916e829bad None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.670853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45508641-f686-4acd-94de-c8916e829bad None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.671149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45508641-f686-4acd-94de-c8916e829bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.681305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45508641-f686-4acd-94de-c8916e829bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.682779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45508641-f686-4acd-94de-c8916e829bad None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.693224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bd6b8-cb31-4202-bdaf-f6b7a8683c03 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-45508641-f686-4acd-94de-c8916e829bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:09.701224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:09.702273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:09.703377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:53:09.703377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:09.703377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:53:09.703377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:09.703377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 538/1098] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:09.708494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45508641-f686-4acd-94de-c8916e829bad None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.711883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.728804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.729678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.729963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.730250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] PATH_INFO: `/v3/users/e0c6c744c81a493fb8f64f714dc58408` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.731490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] RBAC: Authorizing `identity:get_user(user_id=e0c6c744c81a493fb8f64f714dc58408)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.732923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.739180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 561/1099] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/users/e0c6c744c81a493fb8f64f714dc58408 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:09.740129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.769733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.806980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45508641-f686-4acd-94de-c8916e829bad None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.838017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20073a8-d64e-49a4-9f6d-d5494cd051e7 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.840407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.841063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.841346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.841623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.842346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:09.842673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.844049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.855222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 539/1100] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:09.860560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:09.872016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.872666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.872953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.873231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:09.921707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.939594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:09.941567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:09.960174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba9e2-7ade-4bf2-ab41-39f4f8f1c4f1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:09.972197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:09.993938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:09.994684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:09.994977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:09.995258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:10.002439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:10.004527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:53:10.018569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:10.019638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:10.020886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 540/1101] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:53:10.025431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:10.043515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 562/1102] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:10.049145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:10.123605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78508a-0fd3-46d9-97b6-22b7f2548726 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:10.147520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:10.148366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7cec2-0168-4c7c-8dc5-dbc7292a2354 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:10.158663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:10.159330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:10.159615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:10.160426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:10.161222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:10.161801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:10.163256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:10.174502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 541/1103] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:53:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:10.180098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:10.180559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:10.183117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:10.183981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:10.184431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:10.184751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:10.186189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] RBAC: Authorizing `identity:get_project(project_id=643b223c5a7b4f48a0fa7b0278745388)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:10.187657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:10.189364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 563/1104] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:10 2022] GET /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:10.195652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:10.280447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6e779-2acf-4ba1-9a82-2ba310bca8d8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:10.294742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904909-457e-4921-9207-fba42bce9353 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:10.313498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:10.314156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:10.314438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:10.314743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:10.315582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] RBAC: Adding query filter params (role.id=82ae094868e64fd49c4f9c9333153a10, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=a2048d28ca3b42d8b8d82f4754e44433) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:10.316407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:10.317847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a258e69-6281-4365-a84e-a539265ac916 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:10.323757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 542/1105] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:10 2022] GET /identity/v3/role_assignments?user.id=a2048d28ca3b42d8b8d82f4754e44433&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=82ae094868e64fd49c4f9c9333153a10 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:10.327824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:10.328488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:10.328798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:10.329083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:10.329932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=643b223c5a7b4f48a0fa7b0278745388, user.id=e0c6c744c81a493fb8f64f714dc58408) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:10.330251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:10.332387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b839e3-9a69-4b4e-85e4-071afa1a50b6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:10.339299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 564/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:10 2022] GET /identity/v3/role_assignments?user.id=e0c6c744c81a493fb8f64f714dc58408&scope.project.id=643b223c5a7b4f48a0fa7b0278745388&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:53:12.370848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675b084-1afc-453e-b07d-af68e9c5cc28 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:12.371343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675b084-1afc-453e-b07d-af68e9c5cc28 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:12.371431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675b084-1afc-453e-b07d-af68e9c5cc28 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:12.372779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 543/1107] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:12.377934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:12.378352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:12.379127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:12.395428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:12.396605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:12.452865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:12.463254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 565/1108] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:12.488442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675b084-1afc-453e-b07d-af68e9c5cc28 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:12.536609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:12.585838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675b084-1afc-453e-b07d-af68e9c5cc28 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:12.619523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dada3-42ab-468f-95e7-e5225c67c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:12.620292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dada3-42ab-468f-95e7-e5225c67c154 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:12.620577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dada3-42ab-468f-95e7-e5225c67c154 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:12.620850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dada3-42ab-468f-95e7-e5225c67c154 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:12.622000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dada3-42ab-468f-95e7-e5225c67c154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:12.623448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dada3-42ab-468f-95e7-e5225c67c154 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:12.625031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 544/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:12.630499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:12.724637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df46e9e-f37b-4fa9-9c8d-d49acf1e3847 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:12.756990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fac97-2cbf-4e15-b20c-93497193998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:12.757643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fac97-2cbf-4e15-b20c-93497193998b None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:12.757927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fac97-2cbf-4e15-b20c-93497193998b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:12.758203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fac97-2cbf-4e15-b20c-93497193998b None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:12.758954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fac97-2cbf-4e15-b20c-93497193998b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:12.760367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fac97-2cbf-4e15-b20c-93497193998b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:12.780118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 566/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:53:12 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:14.788809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108853-aa9a-41bf-81bc-03560139fbd1 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:14.789335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108853-aa9a-41bf-81bc-03560139fbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:14.789335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13108853-aa9a-41bf-81bc-03560139fbd1 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:14.790754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 545/1111] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:14.795828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835693a7-e395-418b-b2df-a567a04699ac None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:14.796244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835693a7-e395-418b-b2df-a567a04699ac None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:14.796385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835693a7-e395-418b-b2df-a567a04699ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:14.812490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-835693a7-e395-418b-b2df-a567a04699ac None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:14.813658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-835693a7-e395-418b-b2df-a567a04699ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:14.855359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:14.879608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 567/1112] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:14.902869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108853-aa9a-41bf-81bc-03560139fbd1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:14.998933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13108853-aa9a-41bf-81bc-03560139fbd1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:15.014976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:15.032794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:15.033716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:15.034003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:15.034279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] PATH_INFO: `/v3/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:15.035775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] RBAC: Authorizing `identity:get_role(role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:15.037393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:15.038772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 546/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:14 2022] GET /identity/v3/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:15.044375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835693a7-e395-418b-b2df-a567a04699ac None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:15.137628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835693a7-e395-418b-b2df-a567a04699ac None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:15.169331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:15.170171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:15.170450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:15.170753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] PATH_INFO: `/v3/users/4953fa842ddf4c77b6bf7ebe49efeec7` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:15.183688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] RBAC: Authorizing `identity:get_user(user_id=4953fa842ddf4c77b6bf7ebe49efeec7)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:15.185108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:15.186721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 568/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:15 2022] GET /identity/v3/users/4953fa842ddf4c77b6bf7ebe49efeec7 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:15.192689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:15.287215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6326e8-5e51-4220-964a-8a31dfdac172 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:15.320722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:15.321297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:15.321578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:15.321853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:15.371062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:15.387142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:15.388579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:15.482277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 547/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:15.488818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:15.521898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:15.584647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57465f-33ba-4a46-aada-c396aad32d11 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:15.617187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec18841-d437-4733-a1ee-6ff3172684e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:15.618022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec18841-d437-4733-a1ee-6ff3172684e9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:15.618300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec18841-d437-4733-a1ee-6ff3172684e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:15.618569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec18841-d437-4733-a1ee-6ff3172684e9 None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:15.619751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec18841-d437-4733-a1ee-6ff3172684e9 None admin] RBAC: Authorizing `identity:get_project(project_id=643b223c5a7b4f48a0fa7b0278745388)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:15.621164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec18841-d437-4733-a1ee-6ff3172684e9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:15.622843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 569/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:15 2022] GET /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:15.629222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:15.709023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:15.724461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004012d2-0519-4eea-9c7c-d45f99f79083 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:15.756925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:15.757560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:15.757843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:15.758116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:15.758974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] RBAC: Adding query filter params (role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=643b223c5a7b4f48a0fa7b0278745388, user.id=4953fa842ddf4c77b6bf7ebe49efeec7) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:15.759279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:15.760690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda83ac4-877a-4299-91ca-b4ffb0da06d5 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:15.766033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 548/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:15 2022] GET /identity/v3/role_assignments?user.id=4953fa842ddf4c77b6bf7ebe49efeec7&scope.project.id=643b223c5a7b4f48a0fa7b0278745388&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:17.724608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ff687-b83f-4903-afbb-e426db46870e None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.725122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ff687-b83f-4903-afbb-e426db46870e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.725237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ff687-b83f-4903-afbb-e426db46870e None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.726508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 570/1118] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:17.731566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.731936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.732126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.749051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:17.750235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:17.814179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 549/1119] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:17.836053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ff687-b83f-4903-afbb-e426db46870e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:17.934236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ff687-b83f-4903-afbb-e426db46870e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:17.936478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:17.966498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:17.967353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:17.967633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:17.967905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] PATH_INFO: `/v3/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:17.969351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] RBAC: Authorizing `identity:get_role(role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:17.970735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:17.972586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 571/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:17 2022] GET /identity/v3/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:17.978191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.056844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:18.078508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e5907-2915-46e4-9a47-71fcff6774e8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.110805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.111662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.111946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.112221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] PATH_INFO: `/v3/users/4953fa842ddf4c77b6bf7ebe49efeec7` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.113924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] RBAC: Authorizing `identity:get_user(user_id=4953fa842ddf4c77b6bf7ebe49efeec7)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.115387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.116997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 550/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:17 2022] GET /identity/v3/users/4953fa842ddf4c77b6bf7ebe49efeec7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:18.122799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.217867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d0849-12b1-45fc-b7b2-f0022eef393b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.249953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfcc7a-f442-46f6-8edd-a4d0b10f1acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.250762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfcc7a-f442-46f6-8edd-a4d0b10f1acc None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.251043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfcc7a-f442-46f6-8edd-a4d0b10f1acc None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.251316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfcc7a-f442-46f6-8edd-a4d0b10f1acc None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.252463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfcc7a-f442-46f6-8edd-a4d0b10f1acc None admin] RBAC: Authorizing `identity:get_project(project_id=643b223c5a7b4f48a0fa7b0278745388)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.254413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfcc7a-f442-46f6-8edd-a4d0b10f1acc None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.256081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 572/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:18 2022] GET /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:18.261732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:18.356392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c8f21-0f3f-4ca1-b33f-e72a4b3eaeb3 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:18.389576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e9568-ecc2-4ccb-9b9c-5de68c98d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:18.390452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e9568-ecc2-4ccb-9b9c-5de68c98d42a None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:18.390763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e9568-ecc2-4ccb-9b9c-5de68c98d42a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:18.391038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e9568-ecc2-4ccb-9b9c-5de68c98d42a None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388/users/4953fa842ddf4c77b6bf7ebe49efeec7/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:18.393200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e9568-ecc2-4ccb-9b9c-5de68c98d42a None admin] RBAC: Authorizing `identity:create_grant(project_id=643b223c5a7b4f48a0fa7b0278745388, user_id=4953fa842ddf4c77b6bf7ebe49efeec7, role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:18.394673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e9568-ecc2-4ccb-9b9c-5de68c98d42a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:18.407196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 551/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:53:18 2022] PUT /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388/users/4953fa842ddf4c77b6bf7ebe49efeec7/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:20.370853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6878e7f-2371-4c36-8d75-2de50d99be6b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.371367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6878e7f-2371-4c36-8d75-2de50d99be6b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.371472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6878e7f-2371-4c36-8d75-2de50d99be6b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.372764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 573/1124] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:20.377934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.378349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.378502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.389528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:20.395812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:20.397122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:20.461568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 552/1125] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:20.485680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6878e7f-2371-4c36-8d75-2de50d99be6b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.527321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:20.583776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6878e7f-2371-4c36-8d75-2de50d99be6b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.616183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.617015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.617296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.617571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] PATH_INFO: `/v3/roles/a3bccf6d488e41db9572f2b9c15c75b4` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.619061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] RBAC: Authorizing `identity:get_role(role_id=a3bccf6d488e41db9572f2b9c15c75b4)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.620408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.621731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 574/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/roles/a3bccf6d488e41db9572f2b9c15c75b4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:20.627401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.721812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a6cdc-e309-4a49-aac6-6110b7becadb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.754089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.754968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.755249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.755520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] PATH_INFO: `/v3/users/4953fa842ddf4c77b6bf7ebe49efeec7` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.756681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] RBAC: Authorizing `identity:get_user(user_id=4953fa842ddf4c77b6bf7ebe49efeec7)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.758092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:20.759723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 553/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/users/4953fa842ddf4c77b6bf7ebe49efeec7 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:20.765133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:20.860849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a1482-213b-406d-a7aa-3df7c97c0bf9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:20.869867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:20.895853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:20.896482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:20.896762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:20.897034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:20.960450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:20.961866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:21.054332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 575/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:21.060171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:21.063858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:21.155123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe0072-1499-4180-b7f8-646644ceb27b None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:21.186866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50084497-cc68-41a1-82e7-4a1ff45ed0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:21.187696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50084497-cc68-41a1-82e7-4a1ff45ed0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:21.187977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50084497-cc68-41a1-82e7-4a1ff45ed0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:21.188250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50084497-cc68-41a1-82e7-4a1ff45ed0b4 None admin] PATH_INFO: `/v3/projects/643b223c5a7b4f48a0fa7b0278745388` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:21.189397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50084497-cc68-41a1-82e7-4a1ff45ed0b4 None admin] RBAC: Authorizing `identity:get_project(project_id=643b223c5a7b4f48a0fa7b0278745388)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:21.191443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50084497-cc68-41a1-82e7-4a1ff45ed0b4 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:21.193096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 554/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:21 2022] GET /identity/v3/projects/643b223c5a7b4f48a0fa7b0278745388 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:21.198969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:21.208619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:21.294452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a294e-57f4-4a6b-9f95-8c1cd2bd30f5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:21.327602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:21.328246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:21.328529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:21.328805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:21.329627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] RBAC: Adding query filter params (role.id=a3bccf6d488e41db9572f2b9c15c75b4, scope.project.id=643b223c5a7b4f48a0fa7b0278745388, user.id=4953fa842ddf4c77b6bf7ebe49efeec7) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:21.329937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:21.331391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce765c8-1cc9-4545-97c7-78c3a56c34e3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:21.337028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 576/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:21 2022] GET /identity/v3/role_assignments?user.id=4953fa842ddf4c77b6bf7ebe49efeec7&scope.project.id=643b223c5a7b4f48a0fa7b0278745388&role.id=a3bccf6d488e41db9572f2b9c15c75b4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:23.321193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50dff6-8eed-4d90-ab91-b55671a0d82a None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:23.321619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50dff6-8eed-4d90-ab91-b55671a0d82a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:23.321752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50dff6-8eed-4d90-ab91-b55671a0d82a None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:23.323157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 555/1131] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:23.328118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:23.328559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:23.328678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:23.344998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:23.346174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:23.410805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 577/1132] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:23.434105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50dff6-8eed-4d90-ab91-b55671a0d82a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:23.503341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:23.533014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50dff6-8eed-4d90-ab91-b55671a0d82a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:23.566594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18624c5b-edf5-4a41-80c6-32aca4a97fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:23.567394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624c5b-edf5-4a41-80c6-32aca4a97fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:23.567677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624c5b-edf5-4a41-80c6-32aca4a97fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:23.567955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624c5b-edf5-4a41-80c6-32aca4a97fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:23.569112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18624c5b-edf5-4a41-80c6-32aca4a97fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:23.570543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18624c5b-edf5-4a41-80c6-32aca4a97fe2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:23.572160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 556/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:23.577969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:23.580169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:23.673809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18663864-b8c0-4912-844b-e2302ddee3d9 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:23.706554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb76d6a-b4d3-4fe5-9189-8a90a24d19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:23.707230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76d6a-b4d3-4fe5-9189-8a90a24d19e2 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:23.707516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76d6a-b4d3-4fe5-9189-8a90a24d19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:23.707793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb76d6a-b4d3-4fe5-9189-8a90a24d19e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:23.708506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76d6a-b4d3-4fe5-9189-8a90a24d19e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:23.709925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb76d6a-b4d3-4fe5-9189-8a90a24d19e2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:23.733923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 578/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:53:23 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:25.755242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a0584-0f1b-4308-a48c-82f4558221f0 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:25.755793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a0584-0f1b-4308-a48c-82f4558221f0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:25.755793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a0584-0f1b-4308-a48c-82f4558221f0 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:25.757158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 557/1135] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:25.762295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:25.762693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:25.762877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:25.779393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:25.780558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:25.827841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:25.845284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 579/1136] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:25.870018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a0584-0f1b-4308-a48c-82f4558221f0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:25.968014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a0584-0f1b-4308-a48c-82f4558221f0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:26.001461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edbf6a-2d81-410a-80de-a5b3185149f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:26.002296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edbf6a-2d81-410a-80de-a5b3185149f8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:26.002579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edbf6a-2d81-410a-80de-a5b3185149f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:26.002888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edbf6a-2d81-410a-80de-a5b3185149f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:26.004062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edbf6a-2d81-410a-80de-a5b3185149f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:26.005571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edbf6a-2d81-410a-80de-a5b3185149f8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:26.007159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 558/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:53:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:26.012726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:26.107963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1865be-64fd-4605-b0c9-78c1bfeaced3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:26.140235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead05bcd-73b8-4ad0-a580-2e5218b6319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:26.140885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead05bcd-73b8-4ad0-a580-2e5218b6319d None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:26.141165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead05bcd-73b8-4ad0-a580-2e5218b6319d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:26.141438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead05bcd-73b8-4ad0-a580-2e5218b6319d None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:26.142138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead05bcd-73b8-4ad0-a580-2e5218b6319d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:26.143579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead05bcd-73b8-4ad0-a580-2e5218b6319d None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:26.162847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 580/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:53:26 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:28.188607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0088-5c6e-4bd8-a352-0d6419a8736a None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.189128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0088-5c6e-4bd8-a352-0d6419a8736a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.189230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0088-5c6e-4bd8-a352-0d6419a8736a None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.191124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 559/1139] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:28.196172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.196612 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.196713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.213499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:28.214698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:28.246931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:28.280431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 581/1140] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 20:53:28.303527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0088-5c6e-4bd8-a352-0d6419a8736a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:28.314278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:28.401694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0088-5c6e-4bd8-a352-0d6419a8736a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:28.433592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:28.434363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.434733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.435017 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.445302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:28.452770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:28.453939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:28.455282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 560/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:28.460286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:28.564589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49e374-e8cb-4aa8-bc84-ceef5980b28a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:28.596585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:28.597303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.597586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.597857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.598595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:28.598927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:28.600282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:28.607969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 582/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:28.614033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:28.677942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:28.710297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40acf-3ee7-4578-a05d-bd6c1df5682e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:28.742443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:28.743338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.743625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.743904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] PATH_INFO: `/v3/users/d01627da5f1149bdbbb972975c1d3f78` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.757226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] RBAC: Authorizing `identity:get_user(user_id=d01627da5f1149bdbbb972975c1d3f78)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:28.758679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:28.760318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 561/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:28 2022] GET /identity/v3/users/d01627da5f1149bdbbb972975c1d3f78 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:28.765841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:28.796258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:28.862862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3279859-b62a-4e9b-a987-f64a26bccd9b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:28.894755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:28.895384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:28.895668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:28.895943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:28.959770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:28.961237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:28.989861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:29.057447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 583/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:29.063335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:29.158349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb552-3cef-4ec1-ac5a-64dd03d6aa42 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:29.183911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:29.192740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c962ede-9902-4e00-a10e-9594b027e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:29.193581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c962ede-9902-4e00-a10e-9594b027e97a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:29.193863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c962ede-9902-4e00-a10e-9594b027e97a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:29.194137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c962ede-9902-4e00-a10e-9594b027e97a None admin] PATH_INFO: `/v3/projects/bc679ef2e96440bf88e78fe717b012cb` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:29.195268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c962ede-9902-4e00-a10e-9594b027e97a None admin] RBAC: Authorizing `identity:get_project(project_id=bc679ef2e96440bf88e78fe717b012cb)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:29.196691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c962ede-9902-4e00-a10e-9594b027e97a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:29.198720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 562/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:29 2022] GET /identity/v3/projects/bc679ef2e96440bf88e78fe717b012cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:29.204689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:29.300205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa9d52-b00b-40fd-a069-7c5b365e2a9f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:29.327630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:29.333390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:29.334037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:29.334322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:29.334597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:29.335458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=bc679ef2e96440bf88e78fe717b012cb, user.id=d01627da5f1149bdbbb972975c1d3f78) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:29.335770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:29.337192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85faad4-e688-48af-971e-3801b5a524d3 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:29.342543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 584/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:29 2022] GET /identity/v3/role_assignments?user.id=d01627da5f1149bdbbb972975c1d3f78&scope.project.id=bc679ef2e96440bf88e78fe717b012cb&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:31.316137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a88951-aa28-4746-88cd-a853bc69a936 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:31.316599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a88951-aa28-4746-88cd-a853bc69a936 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:31.316696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a88951-aa28-4746-88cd-a853bc69a936 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:31.318068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 563/1147] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:31.323419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:31.323853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:31.323984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:31.340305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:31.341476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:31.405328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 585/1148] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:31.428825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a88951-aa28-4746-88cd-a853bc69a936 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:31.525407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a88951-aa28-4746-88cd-a853bc69a936 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:31.558120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:31.558916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:31.559201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:31.559475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:31.569818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:31.577260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:31.578344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:31.579275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 564/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:31.584165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:31.678804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01a78c-e26f-4337-bc74-71699fee7f26 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:31.703322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:31.712085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:31.712735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:31.713022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:31.713298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:31.714036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:31.714343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:31.715739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:31.723547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 586/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:31.729033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:31.783417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:31.827454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e1774-3910-4d15-a28a-0ad432849846 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:31.858846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:31.859687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:31.859969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:31.860244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] PATH_INFO: `/v3/users/d01627da5f1149bdbbb972975c1d3f78` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:31.861401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] RBAC: Authorizing `identity:get_user(user_id=d01627da5f1149bdbbb972975c1d3f78)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:31.863360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:31.864953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 565/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:31 2022] GET /identity/v3/users/d01627da5f1149bdbbb972975c1d3f78 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:31.870443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:31.966851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ffb9-7f39-446d-9174-612e545f98c2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:31.999047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bda52d-e365-4726-987e-ae0595bf5c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:31.999878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda52d-e365-4726-987e-ae0595bf5c72 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:32.000158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda52d-e365-4726-987e-ae0595bf5c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:32.000431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bda52d-e365-4726-987e-ae0595bf5c72 None admin] PATH_INFO: `/v3/projects/bc679ef2e96440bf88e78fe717b012cb` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:32.001583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bda52d-e365-4726-987e-ae0595bf5c72 None admin] RBAC: Authorizing `identity:get_project(project_id=bc679ef2e96440bf88e78fe717b012cb)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:32.003044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bda52d-e365-4726-987e-ae0595bf5c72 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:32.004677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 587/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:31 2022] GET /identity/v3/projects/bc679ef2e96440bf88e78fe717b012cb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:32.010287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:32.105825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7f7b4-893f-4862-8c6a-26e6a95eeeaf None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:32.127258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:32.138757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e48ed-6446-4a1e-a7fa-47d62be4f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:32.139631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e48ed-6446-4a1e-a7fa-47d62be4f7f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:32.139916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e48ed-6446-4a1e-a7fa-47d62be4f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:32.140191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e48ed-6446-4a1e-a7fa-47d62be4f7f2 None admin] PATH_INFO: `/v3/projects/bc679ef2e96440bf88e78fe717b012cb/users/d01627da5f1149bdbbb972975c1d3f78/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:32.142133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e48ed-6446-4a1e-a7fa-47d62be4f7f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc679ef2e96440bf88e78fe717b012cb, user_id=d01627da5f1149bdbbb972975c1d3f78, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:32.143611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e48ed-6446-4a1e-a7fa-47d62be4f7f2 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:32.156193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 566/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:53:32 2022] PUT /identity/v3/projects/bc679ef2e96440bf88e78fe717b012cb/users/d01627da5f1149bdbbb972975c1d3f78/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:34.139098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a6cc6-3ce9-4c06-9cf9-b861a89c300c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.139632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a6cc6-3ce9-4c06-9cf9-b861a89c300c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.139632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a6cc6-3ce9-4c06-9cf9-b861a89c300c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.141014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 588/1154] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:34.146532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.147008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.147100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.164232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:34.165391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:34.227883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 567/1155] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:34.250353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a6cc6-3ce9-4c06-9cf9-b861a89c300c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:34.307249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:34.351050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a6cc6-3ce9-4c06-9cf9-b861a89c300c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:34.382632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:34.383417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.383698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.383971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.394694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:34.402072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:34.403181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:34.404038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 589/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:34.409197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:34.503275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5919b4-7b40-4e0c-86f7-0998a5dec018 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:34.515767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:34.537642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:34.538301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.538582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.538891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.539627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:34.539930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:34.541292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:34.549671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 568/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:34.555442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:34.651026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39494ecf-7047-4a65-921f-0dc538c29679 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:34.667030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:34.684988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:34.685871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.686155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.686431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] PATH_INFO: `/v3/users/d01627da5f1149bdbbb972975c1d3f78` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.687636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] RBAC: Authorizing `identity:get_user(user_id=d01627da5f1149bdbbb972975c1d3f78)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:34.689054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:34.691230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 590/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:34 2022] GET /identity/v3/users/d01627da5f1149bdbbb972975c1d3f78 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:34.696925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:34.790467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa0b5e-00c2-4fc1-9f72-87fcec5f3909 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:34.823372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:34.824003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:34.824286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:34.824561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:34.864525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:34.889120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:34.890551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:34.984450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 569/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:34.990826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:35.085455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f4317-8779-4e4b-86f0-b88a7126dde4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:35.118308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66b135-33d1-470a-a3ff-8bebe265719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:35.119172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66b135-33d1-470a-a3ff-8bebe265719f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:35.119455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66b135-33d1-470a-a3ff-8bebe265719f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:35.119728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66b135-33d1-470a-a3ff-8bebe265719f None admin] PATH_INFO: `/v3/projects/bc679ef2e96440bf88e78fe717b012cb` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:35.120887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66b135-33d1-470a-a3ff-8bebe265719f None admin] RBAC: Authorizing `identity:get_project(project_id=bc679ef2e96440bf88e78fe717b012cb)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:35.122300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66b135-33d1-470a-a3ff-8bebe265719f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:35.123980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 591/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:34 2022] GET /identity/v3/projects/bc679ef2e96440bf88e78fe717b012cb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:35.129978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:35.201311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:35.225867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bcac6-dc92-4a6b-9b6f-1a98db45df4a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:35.258299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:35.258910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:35.259197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:35.259473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:35.260300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=bc679ef2e96440bf88e78fe717b012cb, user.id=d01627da5f1149bdbbb972975c1d3f78) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:35.260612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:35.262020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96947283-bcd9-4eba-aefd-252474669e4b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:35.268283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 570/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:35 2022] GET /identity/v3/role_assignments?user.id=d01627da5f1149bdbbb972975c1d3f78&scope.project.id=bc679ef2e96440bf88e78fe717b012cb&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:37.281114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9335c9-7ae6-471e-ad2b-a7be678b0e74 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:37.281574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9335c9-7ae6-471e-ad2b-a7be678b0e74 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:37.281668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9335c9-7ae6-471e-ad2b-a7be678b0e74 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:37.283054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 592/1162] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:37.288944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:37.289344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:37.289506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:37.306668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:37.307863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:37.371811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 571/1163] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:37.393496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9335c9-7ae6-471e-ad2b-a7be678b0e74 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:37.445482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:37.491042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9335c9-7ae6-471e-ad2b-a7be678b0e74 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:37.522878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:37.523504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:37.523784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:37.524056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:37.587180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:37.588541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:37.635867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:37.681675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 593/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:37.687490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:37.751093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:37.784411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d834-adb1-4b3d-91a8-290097cadb6b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:37.816745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:37.817529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:37.817816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:37.818095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:37.824620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:37.826054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-48b28936-0c86-4857-8711-d93a1ec5ce3e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:37.840520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:37.841714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:37.842766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 572/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:53:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:37.847986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:37.942377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05526c6d-1fa3-44a6-bb38-4d6ebe68e36c None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:37.974303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:37.974961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:37.975241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:37.975514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:37.976221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:37.976523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:37.977867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74437b5-092a-4398-a00c-5091eef03854 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:37.984640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 594/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Wed Jul 13 20:53:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:39.964426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e135-668a-40dc-a7ce-9cb9b2f7cea1 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.964983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e135-668a-40dc-a7ce-9cb9b2f7cea1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.964983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9e135-668a-40dc-a7ce-9cb9b2f7cea1 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.966330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 573/1167] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:39.971497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61097553-5046-440b-ad11-93d42003c1ab None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:39.971874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61097553-5046-440b-ad11-93d42003c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:39.972049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61097553-5046-440b-ad11-93d42003c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:39.988704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-61097553-5046-440b-ad11-93d42003c1ab None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:39.989858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-61097553-5046-440b-ad11-93d42003c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:40.000140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:40.055741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 595/1168] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:40.078410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9e135-668a-40dc-a7ce-9cb9b2f7cea1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:40.176335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9e135-668a-40dc-a7ce-9cb9b2f7cea1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:40.200825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:40.208601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669a633-8332-4f83-9f9d-a6ef9d69abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:40.209238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669a633-8332-4f83-9f9d-a6ef9d69abd6 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:40.209520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669a633-8332-4f83-9f9d-a6ef9d69abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:40.209792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669a633-8332-4f83-9f9d-a6ef9d69abd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:40.210527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669a633-8332-4f83-9f9d-a6ef9d69abd6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:40.211930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669a633-8332-4f83-9f9d-a6ef9d69abd6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:40.225794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 574/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Wed Jul 13 20:53:40 2022] POST /identity/v3/domains => generated 267 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:53:42.236767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b9e1d-e996-40a7-9478-0e4785782338 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:42.237297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b9e1d-e996-40a7-9478-0e4785782338 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:42.237297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b9e1d-e996-40a7-9478-0e4785782338 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:42.238676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 596/1170] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:42.243804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:42.244245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:42.244367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:42.261460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:42.262663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:42.327585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 575/1171] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:42.350827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b9e1d-e996-40a7-9478-0e4785782338 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:42.447366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b9e1d-e996-40a7-9478-0e4785782338 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:42.479474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c4b6d-5002-4b55-8bd6-594b3764f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:42.480311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c4b6d-5002-4b55-8bd6-594b3764f929 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:42.480590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c4b6d-5002-4b55-8bd6-594b3764f929 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:42.480865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c4b6d-5002-4b55-8bd6-594b3764f929 None admin] PATH_INFO: `/v3/domains/85d6290c794347fe81d72998e6045c4b` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:42.491344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c4b6d-5002-4b55-8bd6-594b3764f929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=85d6290c794347fe81d72998e6045c4b)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:42.492708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c4b6d-5002-4b55-8bd6-594b3764f929 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:42.494279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 597/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Wed Jul 13 20:53:42 2022] GET /identity/v3/domains/85d6290c794347fe81d72998e6045c4b => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:42.499886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:42.601130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f438834-d79f-4705-9218-52a6a4381622 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:42.618007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:42.634958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3f53-b233-4c79-9dac-9be997ec75f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:42.635603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3f53-b233-4c79-9dac-9be997ec75f7 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:42.635883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3f53-b233-4c79-9dac-9be997ec75f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:42.636155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3f53-b233-4c79-9dac-9be997ec75f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:42.636851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3f53-b233-4c79-9dac-9be997ec75f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:42.638252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3f53-b233-4c79-9dac-9be997ec75f7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:42.662544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 576/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:53:42 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:44.699356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67954a0-ac42-44bf-abe6-cbed9751328c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:44.699901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67954a0-ac42-44bf-abe6-cbed9751328c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:44.699901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67954a0-ac42-44bf-abe6-cbed9751328c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:44.701278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 598/1174] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:44.706924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:44.707329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:44.707489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:44.723735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:44.724911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:44.789211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 577/1175] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:44.813557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67954a0-ac42-44bf-abe6-cbed9751328c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:44.823149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:44.913540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67954a0-ac42-44bf-abe6-cbed9751328c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:44.945227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f408f5-e13c-4b62-a04f-a4a0a8dde990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:44.946069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f408f5-e13c-4b62-a04f-a4a0a8dde990 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:44.946347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f408f5-e13c-4b62-a04f-a4a0a8dde990 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:44.946645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f408f5-e13c-4b62-a04f-a4a0a8dde990 None admin] PATH_INFO: `/v3/domains/85d6290c794347fe81d72998e6045c4b` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:44.947801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f408f5-e13c-4b62-a04f-a4a0a8dde990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=85d6290c794347fe81d72998e6045c4b)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:44.949212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f408f5-e13c-4b62-a04f-a4a0a8dde990 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:44.950867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 599/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Wed Jul 13 20:53:44 2022] GET /identity/v3/domains/85d6290c794347fe81d72998e6045c4b => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:44.956867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:45.047460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:45.050921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e959d56-dc57-43d5-a1eb-dc0afef364f1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:45.084939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9324a1-a6f1-4a1e-9ac0-8b5e8d782d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:45.085592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9324a1-a6f1-4a1e-9ac0-8b5e8d782d67 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:45.085873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9324a1-a6f1-4a1e-9ac0-8b5e8d782d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:45.086147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9324a1-a6f1-4a1e-9ac0-8b5e8d782d67 None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:45.086889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9324a1-a6f1-4a1e-9ac0-8b5e8d782d67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:45.088296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9324a1-a6f1-4a1e-9ac0-8b5e8d782d67 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:45.106918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 578/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:53:44 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:53:47.121172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fb850-6201-4678-b59e-7ab4434d727b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.121703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fb850-6201-4678-b59e-7ab4434d727b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.121795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fb850-6201-4678-b59e-7ab4434d727b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.123138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 600/1178] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:47.128666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.129079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.129231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.146147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:47.147365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:47.211965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 579/1179] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:47.234682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fb850-6201-4678-b59e-7ab4434d727b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:47.309020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:47.334362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fb850-6201-4678-b59e-7ab4434d727b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:47.366253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:47.367056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.367341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.367624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.378391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:47.385816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:47.386912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:47.388226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 601/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:47.393057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:47.473677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:47.489582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da06d9-1fe7-4a8e-9cc6-0ed7428439d8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:47.522153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:47.522833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.523119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.523396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.524140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:47.524449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:47.525807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:47.533614 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 580/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:47.539368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:47.635441 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056225a1-1f28-4c0f-874c-4c1ceb582bc3 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:47.669128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:47.669969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.670250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.670517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] PATH_INFO: `/v3/users/9452168ce1834b489f69006f962ec079` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.672406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:47.684955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=9452168ce1834b489f69006f962ec079)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:47.686384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:47.688037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 602/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:47 2022] GET /identity/v3/users/9452168ce1834b489f69006f962ec079 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:47.693664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:47.788487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61c7d-a991-44a2-b6e7-496854c6202f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:47.821168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:47.821803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:47.822091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:47.822373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:47.839697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:47.888128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:47.889508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:47.983259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 581/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:47.989536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:48.088137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ed35b-6997-4d91-a4aa-a98209490ba1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:48.120861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff528233-1931-4eec-820a-83da5dae1eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:48.121701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff528233-1931-4eec-820a-83da5dae1eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:48.122215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff528233-1931-4eec-820a-83da5dae1eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:48.122300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff528233-1931-4eec-820a-83da5dae1eb5 None admin] PATH_INFO: `/v3/projects/2845202f634544edbf98b34d20e5f1ae` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:48.123435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff528233-1931-4eec-820a-83da5dae1eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=2845202f634544edbf98b34d20e5f1ae)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:48.124861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff528233-1931-4eec-820a-83da5dae1eb5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:48.126540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 603/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:47 2022] GET /identity/v3/projects/2845202f634544edbf98b34d20e5f1ae => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:48.132490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:48.179632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:48.228598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20fc68-2eff-4097-b965-a51f623c52ac None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:48.260336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:48.260975 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:48.261258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:48.261532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:48.262359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=2845202f634544edbf98b34d20e5f1ae, user.id=9452168ce1834b489f69006f962ec079) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:48.262694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:48.264112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ca6c3-e59d-4ad7-9334-b465e9ed14ff None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:48.271938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 582/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:48 2022] GET /identity/v3/role_assignments?user.id=9452168ce1834b489f69006f962ec079&scope.project.id=2845202f634544edbf98b34d20e5f1ae&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:50.255918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52653f-b01c-404e-a412-c1e02a7301e8 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.256507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52653f-b01c-404e-a412-c1e02a7301e8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.256507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52653f-b01c-404e-a412-c1e02a7301e8 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.257807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 604/1186] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:50.262951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.263380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.263512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.280725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:50.281888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:50.345296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 583/1187] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:50.367648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52653f-b01c-404e-a412-c1e02a7301e8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.425454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:50.464651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52653f-b01c-404e-a412-c1e02a7301e8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.496504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.497272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.497551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.497823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.508075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:50.515547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:50.516699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:53:50.517947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 605/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:50.522547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.563485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:50.617635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ffc4a-29af-40cb-b726-f53a007b9b89 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.649612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.650267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.650548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.650852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.651588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:50.651891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.653241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.661871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 584/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:50.667498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.762019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525141c-f7b8-4c5e-b51f-942bc940ef46 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.780323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:50.794507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.795382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.795661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.795935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] PATH_INFO: `/v3/users/9452168ce1834b489f69006f962ec079` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.797095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] RBAC: Authorizing `identity:get_user(user_id=9452168ce1834b489f69006f962ec079)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.799086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.800665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 606/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/users/9452168ce1834b489f69006f962ec079 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:50.806269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:50.900917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02cf77-4302-43e3-a1f7-3ff54b9cc907 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:50.903169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:50.934269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ff26d-ae58-45a8-ad35-1c9ec33ead1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:50.935139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ff26d-ae58-45a8-ad35-1c9ec33ead1a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:50.935421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ff26d-ae58-45a8-ad35-1c9ec33ead1a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:50.935690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ff26d-ae58-45a8-ad35-1c9ec33ead1a None admin] PATH_INFO: `/v3/projects/2845202f634544edbf98b34d20e5f1ae` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:50.936846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ff26d-ae58-45a8-ad35-1c9ec33ead1a None admin] RBAC: Authorizing `identity:get_project(project_id=2845202f634544edbf98b34d20e5f1ae)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:50.938243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ff26d-ae58-45a8-ad35-1c9ec33ead1a None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:50.939932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 585/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:50 2022] GET /identity/v3/projects/2845202f634544edbf98b34d20e5f1ae => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:50.945549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:51.041526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134263f4-24d5-47ff-95dc-60fc90d99754 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:51.072962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdf24a-d92c-4352-ad8c-4796ed45b8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:51.073821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf24a-d92c-4352-ad8c-4796ed45b8e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:51.074103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf24a-d92c-4352-ad8c-4796ed45b8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:51.074376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdf24a-d92c-4352-ad8c-4796ed45b8e7 None admin] PATH_INFO: `/v3/projects/2845202f634544edbf98b34d20e5f1ae/users/9452168ce1834b489f69006f962ec079/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:51.076312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf24a-d92c-4352-ad8c-4796ed45b8e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2845202f634544edbf98b34d20e5f1ae, user_id=9452168ce1834b489f69006f962ec079, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:51.077720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdf24a-d92c-4352-ad8c-4796ed45b8e7 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:51.089960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 607/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:53:50 2022] PUT /identity/v3/projects/2845202f634544edbf98b34d20e5f1ae/users/9452168ce1834b489f69006f962ec079/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:53:53.070990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a5db8-253b-4987-8a10-226a57c3582d None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.071517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a5db8-253b-4987-8a10-226a57c3582d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.071517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a5db8-253b-4987-8a10-226a57c3582d None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.072915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 586/1193] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:53:53.078140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.078537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.078720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.095511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:53:53.096670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:53:53.137961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:53.161435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 608/1194] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:53:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:53:53.184382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a5db8-253b-4987-8a10-226a57c3582d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:53.283481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a5db8-253b-4987-8a10-226a57c3582d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:53.317040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:53.317818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.318103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.318380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.328546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:53:53.336159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:53:53.337218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:53:53.338252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 587/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:53:53.343246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:53.437956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3febac-194e-4b75-ba7c-08946476a12c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:53.471577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:53.472240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.472522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.472796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.473539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:53.473842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:53.475245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:53.483217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 609/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:53.489510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:53.520714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:53.587628 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730d672-99b6-4e0c-85ca-75dcf1dce5a5 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:53.619929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:53.620785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.621072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.621350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] PATH_INFO: `/v3/users/9452168ce1834b489f69006f962ec079` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.622527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] RBAC: Authorizing `identity:get_user(user_id=9452168ce1834b489f69006f962ec079)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:53.623974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:53.625579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 588/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Wed Jul 13 20:53:53 2022] GET /identity/v3/users/9452168ce1834b489f69006f962ec079 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:53.631717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:53.672610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:53.729006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffbca6-c6f9-45cf-84d5-bb39c5b099f2 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:53.761302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:53.761933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:53.762218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:53.762494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:53.826880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:53.828316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:53.866896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:53.923182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 610/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:53:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:53:53.929390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:54.022265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:53:54.025450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05ce65-3834-4ef7-b99f-bba98d8a6174 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:54.057903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8848ece-82b1-4b99-8298-c1fecab16a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:54.058775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8848ece-82b1-4b99-8298-c1fecab16a8c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:54.059062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8848ece-82b1-4b99-8298-c1fecab16a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:54.059338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8848ece-82b1-4b99-8298-c1fecab16a8c None admin] PATH_INFO: `/v3/projects/2845202f634544edbf98b34d20e5f1ae` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:54.060504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8848ece-82b1-4b99-8298-c1fecab16a8c None admin] RBAC: Authorizing `identity:get_project(project_id=2845202f634544edbf98b34d20e5f1ae)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:54.061923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8848ece-82b1-4b99-8298-c1fecab16a8c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:54.063622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 589/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Wed Jul 13 20:53:53 2022] GET /identity/v3/projects/2845202f634544edbf98b34d20e5f1ae => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:53:54.069546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:53:54.166185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24073790-9615-456d-a225-9aa1ebd7a1d4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:53:54.198763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:53:54.199405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:53:54.199693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:53:54.199990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:53:54.200823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=2845202f634544edbf98b34d20e5f1ae, user.id=9452168ce1834b489f69006f962ec079) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:53:54.201135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:53:54.202559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad0660-d089-4bd1-8fb5-e8960347abad None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:53:54.208738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 611/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:53:54 2022] GET /identity/v3/role_assignments?user.id=9452168ce1834b489f69006f962ec079&scope.project.id=2845202f634544edbf98b34d20e5f1ae&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:04.053989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=16 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:04.206167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:49.081925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe0f91-bc72-42a8-8c62-57f0cbb9b0cd None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:49.082537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe0f91-bc72-42a8-8c62-57f0cbb9b0cd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:49.082537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe0f91-bc72-42a8-8c62-57f0cbb9b0cd None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:49.084178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 590/1201] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:54:49.090087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:49.090552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:49.090663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:49.108689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:54:49.110205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:49.190043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 612/1202] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:54:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:54:49.215759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe0f91-bc72-42a8-8c62-57f0cbb9b0cd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:49.341641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe0f91-bc72-42a8-8c62-57f0cbb9b0cd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:49.379945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:49.380946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:49.381312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:49.381641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:49.394774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:49.396540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:54:49.419514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:49.421165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:49.422456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:54:49.422456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:54:49.422456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:49.422456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:54:49.422456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 591/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:49.430175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:49.557479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02ab28-4bea-4a8a-8a3c-ba77f6a3c787 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:49.598595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:49.605550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:49.606377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:49.606961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:49.607239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:49.608542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:49.608843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:49.610211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0d913-3d2e-458e-b2ae-56032ae10bfd None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:49.628111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 613/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:49.636356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:49.712725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:49.767958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893689af-70ca-43aa-b6a4-0c0612f40feb None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:49.819271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba198e24-b284-43bd-8d45-6ab84c4e25ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:49.820036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba198e24-b284-43bd-8d45-6ab84c4e25ba None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:49.820367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba198e24-b284-43bd-8d45-6ab84c4e25ba None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:49.820641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba198e24-b284-43bd-8d45-6ab84c4e25ba None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:49.821480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba198e24-b284-43bd-8d45-6ab84c4e25ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:49.823071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba198e24-b284-43bd-8d45-6ab84c4e25ba None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:49.847341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 592/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Wed Jul 13 20:54:49 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:54:52.009107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be879aa7-055f-425e-9328-f032ee7dd621 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:52.009704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be879aa7-055f-425e-9328-f032ee7dd621 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:52.009790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be879aa7-055f-425e-9328-f032ee7dd621 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:52.012078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 614/1206] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:54:52.019075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:52.019684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:52.019684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:52.043739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:54:52.045082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:52.122496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 593/1207] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:54:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:54:52.149358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be879aa7-055f-425e-9328-f032ee7dd621 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:52.262555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be879aa7-055f-425e-9328-f032ee7dd621 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:52.300872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:52.301671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:52.301955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:52.302229 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:52.314104 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:54:52.321719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:52.322782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:52.323692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 615/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:54:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:52.328913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:52.371866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:52.441358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed449fb-c68f-410d-b0e2-25cc8bdf3667 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:52.477069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:52.477745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:52.478032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:52.478308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:52.479085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:52.479410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:52.480771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:52.494657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 594/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:54:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:52.499194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:52.552708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:52.612730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c951-ad15-499f-9545-69d3fe4da9c4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:52.649656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:52.650580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:52.650985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:52.651384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:52.662592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:52.664163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:54:52.682126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:52.683299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:52.684426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:54:52.684426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:54:52.684426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:52.684426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:52.690274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 616/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:52.691553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:52.804796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:52.807939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef448664-4485-40b1-a494-28f4c000a8ff None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:52.844977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:52.845842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:52.846202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:52.846485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:52.847339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:52.847644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:52.849276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:52.863027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 595/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:54:52.869014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:52.993808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea108c1-b741-4ecb-ace6-99c94e90b651 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:52.996108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:53.048655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:53.050309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:53.050603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:53.050965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:53.060648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:53.062064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:54:53.080914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:53.081978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:53.083777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 617/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:53.089195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:53.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fc890-cbde-407f-9918-f0279e3dfe8c None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:53.264858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:53.265638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:53.265981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:53.266291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:53.267271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:53.267586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:53.269098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:53.294690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 596/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Jul 13 20:54:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:53.297058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:53.421566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727ae9-9d88-4ccc-bc0d-486ac679fa07 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:53.465448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:53.466110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:53.466487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:53.466813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:53.508284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:53.553538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:53.555021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:53.674034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 618/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:54:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:54:53.681055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:53.704037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:53.806787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed19e5-8627-4fd3-aee1-aa4353dbfb6f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:53.851302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:53.852167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:53.852518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:53.852858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:53.864543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:53.866008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:54:53.885270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:53.886302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:53.887244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:54:53.887244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:54:53.887244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:53.887244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:54:53.887244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 597/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:53.892466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:53.968487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:54.016856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9937a7-d30f-4a38-98da-0c983177fbf5 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:54.053877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:54.055284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:54.055707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:54.056173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:54.056883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:54.057279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:54.058584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:54.070710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 619/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:54.078497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:54.175992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:54.210863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a24870-9a30-4123-9d71-acf5c34a4fd1 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:54.246199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:54.246942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:54.247227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:54.247505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:54.254166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:54.255634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:54:54.269670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:54.270785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:54:54.271699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:54:54.275913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 598/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:54.276856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:54.392240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34a4c7-dfbd-420e-86d7-d6a098c3d96e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:54.419298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:54.434206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:54.435382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:54.436051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:54.436920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:54.437797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:54.438271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:54.439913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660f491-d843-43cd-8c68-77306cad55e1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:54.455833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 620/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:54.463448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:54.602933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566fb0b-5746-42bf-8d30-385e71044d71 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:54.632472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:54.639517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:54.640270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:54.640469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:54.640769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:54.641575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=ab905149e23347d0aed37819978ac06f) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:54.641875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:54.643335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90642ff8-18b9-426c-9f17-b37412c66722 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:54.648982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 599/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:54:54 2022] GET /identity/v3/role_assignments?user.id=ab905149e23347d0aed37819978ac06f&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:56.952617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712b7b3-265f-44f5-b054-a15caea63418 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:56.953210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712b7b3-265f-44f5-b054-a15caea63418 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:56.953306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712b7b3-265f-44f5-b054-a15caea63418 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:56.955046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 621/1220] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:54:56.961199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:56.961634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:56.961789 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:56.984129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:54:56.985544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:54:57.070661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 600/1221] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:54:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:54:57.106978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712b7b3-265f-44f5-b054-a15caea63418 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:57.223816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712b7b3-265f-44f5-b054-a15caea63418 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:57.263716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:57.265072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:57.265384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:57.265649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:57.276933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:54:57.285630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:57.286797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:57.291548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 622/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:54:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:57.293049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:57.393198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d1750-e059-4215-91c9-1c0939743325 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:57.417318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:57.430043 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:57.430802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:57.431093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:57.431611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:57.432248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] RBAC: Adding query filter params (name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:57.432631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:57.434200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:57.443727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 601/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:54:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:54:57.449896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:57.500904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:57.566353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f3f2f-0ef6-4bc1-80a6-ad26e373946d None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:57.601351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:57.602145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:57.602429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:57.602737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:57.615628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:57.617186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:54:57.635422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:57.636786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:57.637917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:54:57.637917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:54:57.637917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:57.637917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:57.637917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 623/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:57.643145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:57.748349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab187897-800b-4fcb-b14f-976f74581b20 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:57.784288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:57.785125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:57.785526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:57.785948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:57.786809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:57.787283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:57.788778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:57.800458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 602/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:57.806443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:57.903702 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:57.910329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3fd04-df6f-43a1-8cb0-bfa89394bc79 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:57.945219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:57.946153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:57.946455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:57.946770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:57.955819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:57.957522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:54:57.972848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:57.974182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:57.975639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 624/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:54:57.980280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:58.038553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:58.103562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3f8f9-92b1-4429-9b6e-8803f67d4238 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:58.144767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:58.145471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:58.145858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:58.146140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:58.147022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:58.147337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:58.149289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:58.170816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 603/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Jul 13 20:54:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:58.176850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:58.288781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2ec2-6c91-4c27-b519-4e1b018a716a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:58.327578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:58.328451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:58.328774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:58.329255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:58.340031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:58.341496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:54:58.360537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:58.361746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:58.362951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:54:58.362951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:54:58.362951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:54:58.362951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:58.362951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 625/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:54:58.367971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:58.476914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d802c-df16-4318-82f3-6b4315ae826d None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:58.479541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:58.518886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:58.519559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:58.519900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:58.520105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:58.520826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:58.521155 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:58.522506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:58.537094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 604/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:54:58.544172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:58.554971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:58.654815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69119-ddbd-4bef-b3cb-a4626635a193 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:58.691061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425187-d063-4404-a523-ddbafb576422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:58.691981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425187-d063-4404-a523-ddbafb576422 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:58.692264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425187-d063-4404-a523-ddbafb576422 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:58.692549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78425187-d063-4404-a523-ddbafb576422 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:58.699483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425187-d063-4404-a523-ddbafb576422 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:58.700927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78425187-d063-4404-a523-ddbafb576422 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-78425187-d063-4404-a523-ddbafb576422 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:54:58.714854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:54:58.715948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:54:58.717424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 626/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:54:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:54:58.722406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:58.831813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80442d56-f5c9-4444-baff-69a19b2f22c9 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:58.873390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:58.874098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:58.874413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:58.874791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:58.875989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:54:58.876334 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:58.877984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae500-9203-4b22-8f0c-cebcae031ce3 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:58.897778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 605/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:54:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:54:58.898942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425187-d063-4404-a523-ddbafb576422 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:54:58.970319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:54:59.008769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78425187-d063-4404-a523-ddbafb576422 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:54:59.050007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44876ff0-291b-4cf4-a0f2-e2d04556f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:54:59.051276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44876ff0-291b-4cf4-a0f2-e2d04556f47b None admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:54:59.051728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44876ff0-291b-4cf4-a0f2-e2d04556f47b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:54:59.052181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44876ff0-291b-4cf4-a0f2-e2d04556f47b None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/ab905149e23347d0aed37819978ac06f/roles/f19bb6862a9c4537894819cc8db8687f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:54:59.070808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44876ff0-291b-4cf4-a0f2-e2d04556f47b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=ab905149e23347d0aed37819978ac06f, role_id=f19bb6862a9c4537894819cc8db8687f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:54:59.072510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44876ff0-291b-4cf4-a0f2-e2d04556f47b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:54:59.085844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 627/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:54:58 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/ab905149e23347d0aed37819978ac06f/roles/f19bb6862a9c4537894819cc8db8687f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 20:55:01.249749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28caa17-b1f5-4b6f-b873-f5d8534b3ed2 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:01.250285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28caa17-b1f5-4b6f-b873-f5d8534b3ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:01.250285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28caa17-b1f5-4b6f-b873-f5d8534b3ed2 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:01.251683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 606/1233] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:55:01.257375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:01.257919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:01.258016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:01.277973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:01.279440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:01.351214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 628/1234] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:01.377561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28caa17-b1f5-4b6f-b873-f5d8534b3ed2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:01.404558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:01.489564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28caa17-b1f5-4b6f-b873-f5d8534b3ed2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:01.527336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:01.528065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:01.528396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:01.529158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:01.540291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:55:01.548568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:01.549675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:01.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 607/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Wed Jul 13 20:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:01.558321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:01.607458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:01.664202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f0228-b88f-4bac-b867-d178539e58f4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:01.700693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:01.701368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:01.701655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:01.701938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:01.702726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] RBAC: Adding query filter params (name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:01.703037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:01.704411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:01.713304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 629/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Wed Jul 13 20:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:01.721050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:01.817480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:01.834802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619d7ca-f43a-4520-b3d4-a56707cdd557 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:01.872193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:01.873066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:01.873369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:01.873638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:01.885919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:01.887430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:01.907474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:01.908621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:01.909783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:01.909783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:01.909783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:01.909783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:01.909783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 608/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:01.914646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:02.028115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056956af-4b4f-4547-afe0-42989ec6a53b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:02.038738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:02.065934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:02.068165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:02.068396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:02.068686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:02.069385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:02.069694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:02.071089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:02.083534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 630/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:02.089977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:02.207188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfaa2d-d7c0-47b6-984f-86a4df410788 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:02.247287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:02.248119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:02.248406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:02.248775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:02.251550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:02.257708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:02.259379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:55:02.275370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:02.276503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:02.277487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 609/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:02.283591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:02.394807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273996a3-4b1a-4b1d-867a-7e510bb31693 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:02.429561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:02.430527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:02.430867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:02.431165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:02.432078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:02.432414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:02.433900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:02.451947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 631/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Jul 13 20:55:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:02.458035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:02.564108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787669c0-0570-41ac-9b7d-6795fa5d8745 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:02.598687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:02.599336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:02.599618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:02.599894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:02.664302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:02.672542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:02.674094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:02.783269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 610/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:55:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 20:55:02.789556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:02.821326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:02.902077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4120e5-890d-43b3-87f2-abdd5e0697a0 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:02.939985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:02.940752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:02.941058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:02.941310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:02.951982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:02.953523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:02.979947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:02.981510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:02.982669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:02.982669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:02.982669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:02.982669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:02.982669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 632/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:02.987740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:03.098985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:03.105853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e8c03-bc7d-412a-b13a-a37e668c6a61 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:03.141492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:03.142156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:03.142433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:03.142739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:03.143457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:03.143760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:03.145113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:03.156432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 611/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:03.162291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:03.244531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:03.267510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff88026-857d-4209-b5c4-56508b4172d1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:03.306208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:03.307089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:03.307417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:03.307712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:03.314902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:03.316414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:55:03.331073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:03.331978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:03.332876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 633/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:03.338498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:03.448248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20cc20-df37-4685-9d68-5633d1b9c474 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:03.485140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:03.485807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:03.486105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:03.486373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:03.487199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:03.487518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:03.488938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10715543-4457-4a86-8f1d-fe0b08d348e1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:03.501317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 612/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:55:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:03.507726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:03.617721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c2d9-fcf6-4e99-92d0-0163ccc5003f None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:03.655512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:03.656159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:03.656437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:03.656714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:03.657810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] RBAC: Adding query filter params (role.id=f19bb6862a9c4537894819cc8db8687f, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=ab905149e23347d0aed37819978ac06f) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:03.658120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:03.659574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbab98b-41c9-44ab-b7f3-73d63adfa1ba None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:03.665536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 634/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:55:03 2022] GET /identity/v3/role_assignments?user.id=ab905149e23347d0aed37819978ac06f&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=f19bb6862a9c4537894819cc8db8687f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:06.195905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beed0b-1da9-491e-b6e4-95ce71b89334 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:06.196428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beed0b-1da9-491e-b6e4-95ce71b89334 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:06.196428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beed0b-1da9-491e-b6e4-95ce71b89334 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:06.197772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 613/1247] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:55:06.203284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:06.203706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:06.203861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:06.209890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:06.224387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:06.225705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:06.302483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 635/1248] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:06.326254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beed0b-1da9-491e-b6e4-95ce71b89334 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:06.345726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:06.439119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beed0b-1da9-491e-b6e4-95ce71b89334 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:06.475955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:06.476863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:06.477097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:06.477375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:06.489538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:55:06.497713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:06.498731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:06.499719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 614/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:55:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:06.505399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:06.617175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae2d67-c419-4496-8507-099642ad79c0 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:06.650220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:06.658169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:06.658940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:06.659252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:06.659579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:06.660397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:06.660731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:06.662249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:06.671847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 636/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:55:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:06.678254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:06.762578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:06.790213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4642a6-d713-4194-b67d-a44717edb28a None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:06.826992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:06.827808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:06.828100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:06.828380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:06.840702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:06.842283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:06.860905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:06.862123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:06.863249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:06.863249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:06.863249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:06.863249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:06.867926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 615/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:06.869179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:06.978655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef90e2-0681-4c97-97ce-bcdc7cbd666e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:07.016767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:07.017352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:07.017634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:07.017905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:07.018643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:07.018941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:07.020299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:07.033215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 637/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:55:07.039664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:07.145893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b641ba-d533-4ca7-8915-b72c51549fa7 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:07.172186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:07.180167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:07.180977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:07.181255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:07.181531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:07.189941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:07.191413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:55:07.205665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:07.207339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:07.208268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 616/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:07.213403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:07.276113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:07.323671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657cf42-60cd-4af0-9ad6-9bd9269e547b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:07.359409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:07.360080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:07.360387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:07.360641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:07.361484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:07.361790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:07.363262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:07.381237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 638/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Jul 13 20:55:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:07.387693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:07.498247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99001c7-ce1d-448c-b5bb-12c71c906bdb None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:07.532541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:07.533090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:07.533429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:07.533717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:07.589695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:07.602020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:07.603818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:07.710499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 617/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:55:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:55:07.719135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:07.821673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12522-6644-4d6f-b7f7-6285cfa22c6b None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:07.835801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:07.859725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:07.860600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:07.860954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:07.861299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:07.871530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:07.873581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:07.892039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:07.893176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:07.894409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:07.894409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:07.894409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:07.894409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:07.894409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 639/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:07.899236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:07.992139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:08.006280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21dcce-588b-4778-9b9c-abe49198125e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:08.047495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:08.048202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:08.048495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:08.048702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:08.049423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:08.049748 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:08.051153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:08.063697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 618/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:55:08.069832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:08.174855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02363cf9-313a-4a93-b139-57223a204039 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:08.216307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:08.217069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:08.217596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:08.217857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:08.226131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:08.228184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:55:08.244937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:08.246015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:08.247940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 640/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:08.252558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:08.370015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce80b37-9502-4219-b696-e61a5c9f119f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:08.406710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:08.407660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:08.407940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:08.408212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:08.409015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:08.409318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:08.410797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a188b-5369-414d-998a-abd65b6984cb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:08.424777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 619/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:08.431174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:08.458082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:08.536666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac6e2-f344-4a50-ac98-a0c006f18c04 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:08.571993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:08.572657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:08.572937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:08.573211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:08.574065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=ab905149e23347d0aed37819978ac06f) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:08.574372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:08.575910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f937b-638c-4a23-85cf-86723708ea6f None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:08.582114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 641/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:55:08 2022] GET /identity/v3/role_assignments?user.id=ab905149e23347d0aed37819978ac06f&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:10.624948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a40c25-fd45-4db6-9025-5c50d3fcb97b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:10.625489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a40c25-fd45-4db6-9025-5c50d3fcb97b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:10.625489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a40c25-fd45-4db6-9025-5c50d3fcb97b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:10.626903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 620/1261] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:55:10.631891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:10.632287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:10.632448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:10.649243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:10.654786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:10.718663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 642/1262] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:10.741116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a40c25-fd45-4db6-9025-5c50d3fcb97b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:10.752344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:10.840019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a40c25-fd45-4db6-9025-5c50d3fcb97b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:10.872527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:10.873305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:10.873587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:10.873857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:10.883746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:55:10.891780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:10.892910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:10.893768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 621/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 20:55:10.898985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:10.916708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:10.994902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999619-c942-4c4f-9cc6-b4f383697164 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.031060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.032006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.032295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.032646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.033515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:11.033939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.035593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.045908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 643/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:55:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:11.051583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:11.121101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:11.153609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5929a1-2717-44be-9d7d-26b1a8dfc86e None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.186436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.187243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.187523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.187797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.198522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.199986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:11.216344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:11.217392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:11.218329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:11.218329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:11.218329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:11.218329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:11.218329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 622/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:11.223501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:11.303967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:11.322328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a9157-2159-48f5-a6bf-0ad93cae87f0 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.353993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17249454-94cb-4789-a277-6941958008f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.354661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17249454-94cb-4789-a277-6941958008f9 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.354946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17249454-94cb-4789-a277-6941958008f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.355221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17249454-94cb-4789-a277-6941958008f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.355936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17249454-94cb-4789-a277-6941958008f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:11.356251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17249454-94cb-4789-a277-6941958008f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.358121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17249454-94cb-4789-a277-6941958008f9 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.369261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 644/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:11.375059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:11.470331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339ffb3-a73c-4d8f-bb0e-bf72e5187623 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.489227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:11.502641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.503434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.503717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.503994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.511136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.512582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:55:11.526304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:11.527425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:11.529037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 623/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:11.533909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17249454-94cb-4789-a277-6941958008f9 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:11.628498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17249454-94cb-4789-a277-6941958008f9 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.661737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.662387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.662691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.662972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.663795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:11.664097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.665498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.673808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:11.682031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 645/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:11.687581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:11.783803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1b9d-007d-4079-9b5e-46f4adb815b9 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.815121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.815883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.816166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.816442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.827085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.828515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:11.845575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:11.846599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:11.847802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:11.847802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:11.847802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:11.847802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:11.847802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 624/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:11.852236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:11.946298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6336e28-0f60-476e-a79e-b19571eebe38 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:11.978826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:11.979466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:11.979745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:11.980014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:11.980722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:11.981023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:11.982388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:11.993011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 646/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:11.998384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:12.035190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:12.106502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6eb4-d1d9-4f26-ab76-46a530196fbc None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:12.139163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:12.139952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:12.140236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:12.140510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:12.146988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:12.148420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:55:12.162460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:12.163518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:12.164602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 625/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:12.169530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:12.225468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:12.265664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3a83-99c6-4e4c-8471-44433181d9c4 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:12.297956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:12.298542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:12.298850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:12.299121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:12.299903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:12.300211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:12.301621 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c909ac25-5230-49fa-acad-cb1ba5c0c171 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:12.314382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 647/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:12.320469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:12.409071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:12.419274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7f35d-6644-4117-999a-bc3d4425ef3f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:12.452057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679b18d-2a88-404d-9c75-50e849a65586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:12.452925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679b18d-2a88-404d-9c75-50e849a65586 None admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:12.453205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679b18d-2a88-404d-9c75-50e849a65586 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:12.453476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679b18d-2a88-404d-9c75-50e849a65586 None admin] PATH_INFO: `/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/ab905149e23347d0aed37819978ac06f/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:12.455406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679b18d-2a88-404d-9c75-50e849a65586 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0da876cd37f4d3a9e0d128652841317, user_id=ab905149e23347d0aed37819978ac06f, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:12.456847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679b18d-2a88-404d-9c75-50e849a65586 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:12.468609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 626/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Wed Jul 13 20:55:12 2022] PUT /identity/v3/projects/c0da876cd37f4d3a9e0d128652841317/users/ab905149e23347d0aed37819978ac06f/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 20:55:14.436424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad067-7c08-4863-a3c3-d439e50ea982 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:14.436955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad067-7c08-4863-a3c3-d439e50ea982 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:14.436955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ad067-7c08-4863-a3c3-d439e50ea982 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:14.438320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 648/1274] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:55:14.443295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:14.443683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:14.443851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:14.461201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:14.462366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:14.525283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 627/1275] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:14.548100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ad067-7c08-4863-a3c3-d439e50ea982 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:14.644912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ad067-7c08-4863-a3c3-d439e50ea982 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:14.661472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:14.678011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:14.678810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:14.679093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:14.679364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:14.689527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 20:55:14.696996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:14.698064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:14.699329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 649/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Wed Jul 13 20:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:14.703986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:14.782385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:14.800896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b6fab-7648-4ef6-b537-f6f5f61c6f04 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:14.834195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:14.834887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:14.835174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:14.835455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:14.836204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:14.836512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:14.837879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:14.845852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 628/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Wed Jul 13 20:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:14.851433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:14.947627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8936d80e-3aaa-40f0-943b-ab0718b13814 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:14.978785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:14.979551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:14.979833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:14.980108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:14.990640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:14.992077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:15.008886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:15.009891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:15.011221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:15.011221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:15.011221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:15.011221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:15.011221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 650/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:15.016672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:15.124096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4b62c-2342-41c2-b1d4-9fb3dae2910f None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:15.155846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:15.156755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:15.157049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:15.157324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:15.158040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:15.158345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:15.159737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:15.171695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 629/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 20:55:15.177286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:15.206672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:15.273422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeb408-49b9-4a16-915f-2057bd2b29d1 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:15.306894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:15.307677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:15.307959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:15.308233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:15.315433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:15.316867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 20:55:15.330551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:15.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:15.332620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 651/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:15.337362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:15.341556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:15.433553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778c398-50d7-43de-b801-b0dfccdfb657 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:15.466236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d358e-deac-429a-b690-cda1290905af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:15.466922 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d358e-deac-429a-b690-cda1290905af None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:15.467208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d358e-deac-429a-b690-cda1290905af None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:15.467486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d358e-deac-429a-b690-cda1290905af None admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:15.468312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d358e-deac-429a-b690-cda1290905af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:15.468619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d358e-deac-429a-b690-cda1290905af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:15.470025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d358e-deac-429a-b690-cda1290905af None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:15.486313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 630/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Wed Jul 13 20:55:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:15.491918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:15.579797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:15.589168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfdec6-e274-45c0-98d3-352167e39732 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:15.621150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:15.621773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:15.622052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:15.622352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:15.686438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:15.687834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:15.771673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:15.782582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 652/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Wed Jul 13 20:55:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 20:55:15.788203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d358e-deac-429a-b690-cda1290905af None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:15.846745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:15.884512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d358e-deac-429a-b690-cda1290905af None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:15.917648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:15.918425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:15.918755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:15.919034 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:15.928659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:15.930099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 20:55:15.948240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:15.949106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:15.950652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 20:55:15.950652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 20:55:15.950652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 20:55:15.950652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:15.950652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 631/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:15.955520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:16.056307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bd21f-73e9-4ecc-9f8f-809b79cf5405 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:16.089348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:16.089996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:16.090278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:16.090584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:16.091335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:16.091640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:16.092998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:16.103822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 653/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Wed Jul 13 20:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:16.109342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:16.205689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248de59-cb05-47d8-bceb-6874c7d59697 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:16.219097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:16.239079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:16.239855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:16.240142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:16.240417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:16.247491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:16.248940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 20:55:16.262404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:16.263443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: ERROR keystone.server.flask.application Jul 13 20:55:16.264670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 632/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Wed Jul 13 20:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:16.269265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:16.318394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:16.366479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f6cf9-e0ac-40e5-a364-9c21e3a9339a None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:16.398583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:16.399270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:16.399535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:16.399811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:16.400592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:16.400901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:16.402328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076844c-5177-484c-bd87-97e2f10d0b7a None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:16.414227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 654/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Wed Jul 13 20:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:16.420564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:16.517099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a64f13f-6f08-4539-932a-ba150829e8c8 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:16.550180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:16.550846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:16.551132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:16.551408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:16.552235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] RBAC: Adding query filter params (role.id=685cce0e789945d8ad7e2d0fab42e194, scope.project.id=c0da876cd37f4d3a9e0d128652841317, user.id=ab905149e23347d0aed37819978ac06f) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:16.552548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:16.553971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302a1c5-90f2-4a44-8a4c-4be1cf5856e6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:16.559674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 633/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Wed Jul 13 20:55:16 2022] GET /identity/v3/role_assignments?user.id=ab905149e23347d0aed37819978ac06f&scope.project.id=c0da876cd37f4d3a9e0d128652841317&role.id=685cce0e789945d8ad7e2d0fab42e194 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:18.571187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3ed04-1092-4076-ba58-bb62ea256d27 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:18.571699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3ed04-1092-4076-ba58-bb62ea256d27 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:18.571699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3ed04-1092-4076-ba58-bb62ea256d27 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:18.573012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 655/1288] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:55:18.578784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:18.579149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:18.579349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:18.596212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:18.597420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:18.625636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:18.662881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 634/1289] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:18.684502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3ed04-1092-4076-ba58-bb62ea256d27 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:18.782470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3ed04-1092-4076-ba58-bb62ea256d27 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:18.792651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:18.815942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:18.816716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:18.816995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:18.817268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:18.817916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:18.819299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:18.831344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:18.832545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:18.833728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 656/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:55:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:18.838210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:18.933401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc31b3-5cac-47a5-9c5a-5f7db2970f2b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:18.967099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:18.967750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:18.968036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:18.968314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:18.969017 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:18.969322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:18.970702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8666-b3ec-4f1c-b5dc-0a56e4a7119b None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:18.975407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 635/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Jul 13 20:55:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:18.980817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:19.087736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7400be8-edca-4afe-8006-0d38151bf070 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:19.121241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:19.121926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:19.122213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:19.122494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] PATH_INFO: `/v3/services` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:19.123238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:19.123545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:19.124912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a3247-eedb-4d8a-a56f-641e5cd64024 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:19.129269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 657/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Jul 13 20:55:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:21.106881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81623c-e351-4513-87dc-567241953cdf None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:21.107536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81623c-e351-4513-87dc-567241953cdf None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:21.107536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81623c-e351-4513-87dc-567241953cdf None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:21.108809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 636/1293] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:55:21.113987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8dda-a37f-471e-9e86-6a8bd788202d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:21.114398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8dda-a37f-471e-9e86-6a8bd788202d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:21.114548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e8dda-a37f-471e-9e86-6a8bd788202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:21.131035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-2e2e8dda-a37f-471e-9e86-6a8bd788202d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:21.132213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2e2e8dda-a37f-471e-9e86-6a8bd788202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:21.161127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:21.198137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 658/1294] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:21.220970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81623c-e351-4513-87dc-567241953cdf None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:21.251272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:21.320186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81623c-e351-4513-87dc-567241953cdf None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:21.352638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd667d4c-fde5-4868-b398-685637881ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:21.353291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd667d4c-fde5-4868-b398-685637881ae6 None admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:21.353574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd667d4c-fde5-4868-b398-685637881ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:21.353850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd667d4c-fde5-4868-b398-685637881ae6 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:21.354577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd667d4c-fde5-4868-b398-685637881ae6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:21.356505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd667d4c-fde5-4868-b398-685637881ae6 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:21.369541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 637/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Wed Jul 13 20:55:21 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 20:55:23.366240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c246-9e68-4b8e-91c2-d2fa7fe0576b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:23.366758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c246-9e68-4b8e-91c2-d2fa7fe0576b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:23.366855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c246-9e68-4b8e-91c2-d2fa7fe0576b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:23.368370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 659/1296] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:55:23.373389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:23.373769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:23.373950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:23.390211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:23.391418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:23.461485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 638/1297] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:23.484315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c246-9e68-4b8e-91c2-d2fa7fe0576b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:23.582693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c246-9e68-4b8e-91c2-d2fa7fe0576b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:23.615581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:23.616359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:23.616640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:23.616913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:23.617559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:23.618940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:23.630341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:23.631624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:23.632706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 660/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:55:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:23.637827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:23.655974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:23.733686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6791a-5aae-457c-88e7-b4aa48a88af2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:23.765695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:23.766342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:23.766646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:23.766927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:23.767625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:23.767931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:23.769277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:23.774282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 639/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Jul 13 20:55:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:23.780536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:23.810583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:23.876115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693081ae-0d63-49dd-8d25-c7bbb6c8410e None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:23.908898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:23.909534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:23.909811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:23.910082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:23.910889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] RBAC: Adding query filter params (interface=public, service_id=00d32ea6a5b3445aafae0f276e027124, region_id=RegionOne) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:23.911198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:23.912534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055f5f9-9aa3-4313-abcb-4e9df38ca839 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:23.917346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 661/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Wed Jul 13 20:55:23 2022] GET /identity/v3/endpoints?service_id=00d32ea6a5b3445aafae0f276e027124&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:23.922709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:23.999681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:24.019403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2f415-13b1-4937-bcce-830b47a9aa30 None admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:24.062340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620563c-7a9f-4328-9e6d-6085c22b99e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:24.063102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620563c-7a9f-4328-9e6d-6085c22b99e1 None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:24.063412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620563c-7a9f-4328-9e6d-6085c22b99e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:24.063722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620563c-7a9f-4328-9e6d-6085c22b99e1 None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:24.064519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620563c-7a9f-4328-9e6d-6085c22b99e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:24.066172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620563c-7a9f-4328-9e6d-6085c22b99e1 None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:24.073893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 640/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Wed Jul 13 20:55:23 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:55:26.057519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba3662-7650-412a-9353-b21e7cafd5c8 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:26.058069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba3662-7650-412a-9353-b21e7cafd5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:26.058069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba3662-7650-412a-9353-b21e7cafd5c8 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:26.060035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 662/1302] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:55:26.065059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:26.065448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:26.065627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:26.081930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:55:26.083128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:55:26.147004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 641/1303] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:55:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 20:55:26.169875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba3662-7650-412a-9353-b21e7cafd5c8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:26.260909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:26.268013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba3662-7650-412a-9353-b21e7cafd5c8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:26.301296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:26.302079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:26.302361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:26.302656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:26.303303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:26.304650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 20:55:26.316233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 20:55:26.317407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application with Lock( Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self._enter() Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application return self.creator() Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application created_value = creator( Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: ERROR keystone.server.flask.application Jul 13 20:55:26.318286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 663/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Wed Jul 13 20:55:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 20:55:26.323356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:26.395323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:26.417612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c301659-77ed-48ef-9c89-67c23bc0ffa0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:26.450113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:26.450790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:26.451074 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:26.451347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] PATH_INFO: `/v3/services` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:26.452042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 20:55:26.452343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:26.453693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59965e67-cfe3-4463-b248-223ca084b3bb None admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:26.458673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 642/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Wed Jul 13 20:55:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 20:55:26.464262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:55:26.560543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6332c-deae-47d5-88e7-ba0031af7df6 None admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:55:26.592137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfba5f2-c02c-44ff-98a6-17b9c6a7c981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:55:26.592709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfba5f2-c02c-44ff-98a6-17b9c6a7c981 None admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:55:26.592987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfba5f2-c02c-44ff-98a6-17b9c6a7c981 None admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:55:26.593257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfba5f2-c02c-44ff-98a6-17b9c6a7c981 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:55:26.593999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfba5f2-c02c-44ff-98a6-17b9c6a7c981 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:55:26.595365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfba5f2-c02c-44ff-98a6-17b9c6a7c981 None admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:55:26.613081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 664/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Wed Jul 13 20:55:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 20:55:36.455695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=30 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:55:36.608507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:29.099770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c24180-b6f9-4818-8bcf-42d5d84c9e71 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:29.100314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c24180-b6f9-4818-8bcf-42d5d84c9e71 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:29.100314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c24180-b6f9-4818-8bcf-42d5d84c9e71 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:29.106095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 643/1307] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:29.107507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0572-d759-4bc1-888c-ba69a936485f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:29.107959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0572-d759-4bc1-888c-ba69a936485f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:29.108056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e0572-d759-4bc1-888c-ba69a936485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:29.189532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9b3e0572-d759-4bc1-888c-ba69a936485f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:29.191208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9b3e0572-d759-4bc1-888c-ba69a936485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:29.379055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 665/1308] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:29.430037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472898c2-fc0d-4e1b-8d57-dc46fb6f5302 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:29.430329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472898c2-fc0d-4e1b-8d57-dc46fb6f5302 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:29.430610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472898c2-fc0d-4e1b-8d57-dc46fb6f5302 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:29.432142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 644/1309] 10.222.0.39 () {54 vars in 964 bytes} [Wed Jul 13 20:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:29.437888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abf62f-e78d-4611-a4fe-8d069c8b6233 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:29.438449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abf62f-e78d-4611-a4fe-8d069c8b6233 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:29.438998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abf62f-e78d-4611-a4fe-8d069c8b6233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:29.534259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-90abf62f-e78d-4611-a4fe-8d069c8b6233 None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:29.535851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-90abf62f-e78d-4611-a4fe-8d069c8b6233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:29.561727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:29.629210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 666/1310] 10.222.0.39 () {58 vars in 1061 bytes} [Wed Jul 13 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:29.638562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472898c2-fc0d-4e1b-8d57-dc46fb6f5302 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:29.640401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-472898c2-fc0d-4e1b-8d57-dc46fb6f5302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:29.760171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472898c2-fc0d-4e1b-8d57-dc46fb6f5302 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:29.835621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:29.836306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:29.836585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:29.836855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:29.838494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:29.894986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:29.952352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:29.953788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa206a71-7f33-41cc-86bd-305730e6df6c service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:29.974880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 645/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:32.288761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd88c3-ee79-4c8f-bc15-691f48c22a1f None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:32.289958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd88c3-ee79-4c8f-bc15-691f48c22a1f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:32.289958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbd88c3-ee79-4c8f-bc15-691f48c22a1f None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:32.295710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 667/1312] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:32.297451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0633d8-0136-408b-a17a-c303bcfe30d1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:32.297944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0633d8-0136-408b-a17a-c303bcfe30d1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:32.298036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0633d8-0136-408b-a17a-c303bcfe30d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:32.388237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9d0633d8-0136-408b-a17a-c303bcfe30d1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:32.389385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9d0633d8-0136-408b-a17a-c303bcfe30d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:32.491206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 646/1313] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:32.542078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862b09-2245-408c-8c96-cf1ec755ac57 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:32.542407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862b09-2245-408c-8c96-cf1ec755ac57 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:32.542718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb862b09-2245-408c-8c96-cf1ec755ac57 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:32.544113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 668/1314] 10.222.0.39 () {54 vars in 964 bytes} [Wed Jul 13 20:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:32.550059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6d7f0-a6b4-4617-8142-f4b494091af9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:32.550465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6d7f0-a6b4-4617-8142-f4b494091af9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:32.550788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d6d7f0-a6b4-4617-8142-f4b494091af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:32.562222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:32.635540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e8d6d7f0-a6b4-4617-8142-f4b494091af9 None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:32.636760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e8d6d7f0-a6b4-4617-8142-f4b494091af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:32.719262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 647/1315] 10.222.0.39 () {58 vars in 1061 bytes} [Wed Jul 13 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:32.728195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862b09-2245-408c-8c96-cf1ec755ac57 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:32.729950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cb862b09-2245-408c-8c96-cf1ec755ac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:32.835473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb862b09-2245-408c-8c96-cf1ec755ac57 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:32.906866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:32.909687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:32.910379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:32.910688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:32.911014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:32.913223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:33.019537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:33.020968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1c739-ab88-416f-8e7d-497f542da4ee service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:33.041200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 669/1316] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:56:35.566432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dc929-59a0-4d57-ba53-8b32538dc5e2 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:35.566987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dc929-59a0-4d57-ba53-8b32538dc5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:35.566987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dc929-59a0-4d57-ba53-8b32538dc5e2 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:35.568365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 648/1317] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:35.574028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6599351-471b-4c94-8e2b-792d49e0e391 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:35.574424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6599351-471b-4c94-8e2b-792d49e0e391 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:35.574573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6599351-471b-4c94-8e2b-792d49e0e391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:35.636955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e6599351-471b-4c94-8e2b-792d49e0e391 None None] MFA Rules not processed for user `97ea2dd3c47c492b95e460c97de31345`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:35.638129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e6599351-471b-4c94-8e2b-792d49e0e391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:35.687407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:35.730761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 670/1318] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:35.769783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dc929-59a0-4d57-ba53-8b32538dc5e2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:35.771378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4f7dc929-59a0-4d57-ba53-8b32538dc5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:35.877445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dc929-59a0-4d57-ba53-8b32538dc5e2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:35.880286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:35.950332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:35.951007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:35.951293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:35.951571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:35.953206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:36.058470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:36.059929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99af02e-6e3c-42b4-8aef-5fc7fd396288 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:36.078595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 649/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:38.596994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd90cfd-5357-40d6-b281-2cbdcb442b2d None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:38.597499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd90cfd-5357-40d6-b281-2cbdcb442b2d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:38.597499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd90cfd-5357-40d6-b281-2cbdcb442b2d None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:38.598867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 671/1320] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:38.603951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3645d8-59e0-4d45-844d-4d403e0525f9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:38.604389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3645d8-59e0-4d45-844d-4d403e0525f9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:38.604521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3645d8-59e0-4d45-844d-4d403e0525f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:38.643403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:38.669002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fc3645d8-59e0-4d45-844d-4d403e0525f9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:38.670199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fc3645d8-59e0-4d45-844d-4d403e0525f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:38.763998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 650/1321] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:38.801330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd90cfd-5357-40d6-b281-2cbdcb442b2d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:38.802945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cfd90cfd-5357-40d6-b281-2cbdcb442b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:38.903344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd90cfd-5357-40d6-b281-2cbdcb442b2d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:38.973865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:38.974507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:38.974824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:38.975101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:38.976955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:38.984109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:39.086838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:39.088260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87df2e-f32c-4468-9d44-baed36c36e40 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:39.106170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 672/1322] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:41.747338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cf168-15e4-4820-9d24-a15948e06f74 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:41.747938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cf168-15e4-4820-9d24-a15948e06f74 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:41.747938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501cf168-15e4-4820-9d24-a15948e06f74 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:41.749302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 651/1323] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:41.754377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb723a4-d331-436e-a446-fc850cb80c33 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:41.754789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb723a4-d331-436e-a446-fc850cb80c33 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:41.754958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb723a4-d331-436e-a446-fc850cb80c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:41.819671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-3eb723a4-d331-436e-a446-fc850cb80c33 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:41.822002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3eb723a4-d331-436e-a446-fc850cb80c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:41.878436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:41.917835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 673/1324] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:41.954093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cf168-15e4-4820-9d24-a15948e06f74 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:41.955692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-501cf168-15e4-4820-9d24-a15948e06f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:42.065158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501cf168-15e4-4820-9d24-a15948e06f74 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:42.078610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:42.145000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:42.145657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:42.145940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:42.146217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:42.148463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:42.255539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:42.256982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f316f-a4c6-400f-9728-f280ee7210b6 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:42.277088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 652/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:44.331954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba881f5-50af-4b79-9c7b-7b2fecc2e8bb None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:44.332495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba881f5-50af-4b79-9c7b-7b2fecc2e8bb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:44.332495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba881f5-50af-4b79-9c7b-7b2fecc2e8bb None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:44.333874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 674/1326] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:44.339236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36af84-c317-485f-b761-58e452117f0b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:44.339635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36af84-c317-485f-b761-58e452117f0b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:44.339800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36af84-c317-485f-b761-58e452117f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:44.386146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:44.404221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2d36af84-c317-485f-b761-58e452117f0b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:44.405320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2d36af84-c317-485f-b761-58e452117f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:44.501564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 653/1327] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:44.536862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba881f5-50af-4b79-9c7b-7b2fecc2e8bb None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:44.538926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bba881f5-50af-4b79-9c7b-7b2fecc2e8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:44.639000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba881f5-50af-4b79-9c7b-7b2fecc2e8bb None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:44.711387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:44.712038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:44.712316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:44.712585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:44.714196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:44.731395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:44.826906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:44.828398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe4bbb-8075-4409-92b5-ba60035a6f64 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:44.847786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 675/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:46.922807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682654f4-356d-41ca-99bb-d208f6218dbd None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:46.923343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682654f4-356d-41ca-99bb-d208f6218dbd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:46.923461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682654f4-356d-41ca-99bb-d208f6218dbd None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:46.924776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 654/1329] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:56:46.930474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c8870-c3d8-4147-a25f-e977259f883b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:46.930924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c8870-c3d8-4147-a25f-e977259f883b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:46.931071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c8870-c3d8-4147-a25f-e977259f883b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:46.996993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1a8c8870-c3d8-4147-a25f-e977259f883b None None] MFA Rules not processed for user `97ea2dd3c47c492b95e460c97de31345`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:46.998609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1a8c8870-c3d8-4147-a25f-e977259f883b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:47.063348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:47.098755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 676/1330] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:47.135807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682654f4-356d-41ca-99bb-d208f6218dbd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:47.137366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-682654f4-356d-41ca-99bb-d208f6218dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:47.244288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682654f4-356d-41ca-99bb-d208f6218dbd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:47.264088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:47.321819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:47.322462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:47.322775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:47.323054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:47.324696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:47.434041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:47.435527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fd5d8-0a19-4a5a-8cfd-ba8e55584923 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:47.460156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 655/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:49.905958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe809ec-f47d-45fd-977b-dd00b4c585cf None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:49.906513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe809ec-f47d-45fd-977b-dd00b4c585cf None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:49.906513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe809ec-f47d-45fd-977b-dd00b4c585cf None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:49.907899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 677/1332] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:49.913103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482712-65e4-4709-9c1a-1807a098052e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:49.913487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482712-65e4-4709-9c1a-1807a098052e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:49.913694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482712-65e4-4709-9c1a-1807a098052e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:49.975309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:49.978579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9f482712-65e4-4709-9c1a-1807a098052e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:49.979924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9f482712-65e4-4709-9c1a-1807a098052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:50.076075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 656/1333] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:50.110790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe809ec-f47d-45fd-977b-dd00b4c585cf None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:50.112359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fbe809ec-f47d-45fd-977b-dd00b4c585cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:50.212646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe809ec-f47d-45fd-977b-dd00b4c585cf None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:50.283350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:50.284000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:50.284283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:50.284560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:50.286192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:50.312870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:50.392331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:50.393748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989a2ca-3309-465e-9d18-925680254b1a service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:50.411303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 678/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:52.444843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f53a00-f260-4f5c-89cb-2e46a811eb73 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:52.445419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f53a00-f260-4f5c-89cb-2e46a811eb73 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:52.445419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f53a00-f260-4f5c-89cb-2e46a811eb73 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:52.446813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 657/1335] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:52.451750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68648cd6-606d-46db-88e2-939245640e73 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:52.452116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68648cd6-606d-46db-88e2-939245640e73 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:52.452304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68648cd6-606d-46db-88e2-939245640e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:52.514370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-68648cd6-606d-46db-88e2-939245640e73 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:52.515575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-68648cd6-606d-46db-88e2-939245640e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:52.587411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:52.609630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 679/1336] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:52.648795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f53a00-f260-4f5c-89cb-2e46a811eb73 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:52.650357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-11f53a00-f260-4f5c-89cb-2e46a811eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:52.751178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f53a00-f260-4f5c-89cb-2e46a811eb73 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:52.786562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:52.830870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:52.831521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:52.831805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:52.832080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:52.833721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:52.940080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:52.941525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e695cc-7d7d-4817-a6c0-a05dd875953d service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:52.960621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 658/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:55.275986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd736f-a53b-41aa-a51f-7b86603620f8 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:55.276413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd736f-a53b-41aa-a51f-7b86603620f8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:55.276413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd736f-a53b-41aa-a51f-7b86603620f8 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:55.278004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 680/1338] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:55.283273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272469c1-ea82-43c0-b3c3-fdd33e735002 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:55.283671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272469c1-ea82-43c0-b3c3-fdd33e735002 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:55.283827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272469c1-ea82-43c0-b3c3-fdd33e735002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:55.346181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-272469c1-ea82-43c0-b3c3-fdd33e735002 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:55.347394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-272469c1-ea82-43c0-b3c3-fdd33e735002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:55.355523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:55.443772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 659/1339] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:56:55.479105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd736f-a53b-41aa-a51f-7b86603620f8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:55.480613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2cfd736f-a53b-41aa-a51f-7b86603620f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:55.583504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd736f-a53b-41aa-a51f-7b86603620f8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:56:55.653527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:56:55.654167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:55.654448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:55.654760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:55.656375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:55.690956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:55.762436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:56:55.763965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761893b-7a2c-4d0b-8817-f62da12b483a service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:56:55.781698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 681/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:56:59.765161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd03f4-08b1-495e-8cc5-5f2341ea2ced None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:59.765660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd03f4-08b1-495e-8cc5-5f2341ea2ced None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:59.765660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cd03f4-08b1-495e-8cc5-5f2341ea2ced None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:59.767108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 660/1341] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:56:59.772856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b8a42-26ab-4c74-8e17-e9b459f1c823 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:56:59.773284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b8a42-26ab-4c74-8e17-e9b459f1c823 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:56:59.773405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b8a42-26ab-4c74-8e17-e9b459f1c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:56:59.839590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1d6b8a42-26ab-4c74-8e17-e9b459f1c823 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:56:59.840783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1d6b8a42-26ab-4c74-8e17-e9b459f1c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:56:59.927852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:56:59.939713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 682/1342] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:56:59.978398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd03f4-08b1-495e-8cc5-5f2341ea2ced None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:56:59.979972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a4cd03f4-08b1-495e-8cc5-5f2341ea2ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:00.087888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cd03f4-08b1-495e-8cc5-5f2341ea2ced None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:00.125225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:00.162231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:00.162919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:00.163205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:00.163480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:00.165142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:00.273919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:00.275401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e3829-ef61-4dbc-9e69-f01d27be5e99 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:00.294572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 661/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:02.929386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fe8bf-bcc7-42cf-808c-facd954d40cb None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:02.929956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fe8bf-bcc7-42cf-808c-facd954d40cb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:02.929956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fe8bf-bcc7-42cf-808c-facd954d40cb None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:02.931318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 683/1344] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:02.936956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba222a-42c7-4c68-814f-3cb4cd837ba0 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:02.937460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba222a-42c7-4c68-814f-3cb4cd837ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:02.937590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba222a-42c7-4c68-814f-3cb4cd837ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:03.007345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-feba222a-42c7-4c68-814f-3cb4cd837ba0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:03.008491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-feba222a-42c7-4c68-814f-3cb4cd837ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:03.027851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:03.107584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 662/1345] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:57:03.144183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fe8bf-bcc7-42cf-808c-facd954d40cb None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:03.145709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c16fe8bf-bcc7-42cf-808c-facd954d40cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:03.249482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fe8bf-bcc7-42cf-808c-facd954d40cb None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:03.328270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:03.328925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:03.329204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:03.329474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:03.331542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:03.375611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:03.440374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:03.441804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350a4b9-1e83-4cad-a993-90aaaa403b9c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:03.459400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 684/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:07.408131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715febe0-bbcc-485c-b646-feb9f4af1615 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:07.408688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715febe0-bbcc-485c-b646-feb9f4af1615 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:07.408688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715febe0-bbcc-485c-b646-feb9f4af1615 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:07.410057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 663/1347] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:07.415264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb50a-d84f-4d24-ab60-9dfe940fda96 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:07.415700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb50a-d84f-4d24-ab60-9dfe940fda96 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:07.415827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb50a-d84f-4d24-ab60-9dfe940fda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:07.480041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a28fb50a-d84f-4d24-ab60-9dfe940fda96 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:07.481264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a28fb50a-d84f-4d24-ab60-9dfe940fda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:07.592752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:07.595381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 685/1348] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:07.633610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715febe0-bbcc-485c-b646-feb9f4af1615 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:07.635402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-715febe0-bbcc-485c-b646-feb9f4af1615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:07.750437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715febe0-bbcc-485c-b646-feb9f4af1615 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:07.799818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:07.826924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:07.827582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:07.827862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:07.828138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:07.829817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:07.944954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:07.946410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f780423-e484-4704-8503-09d8eba99e2f service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:07.965313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 664/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:12.854727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497909a9-6874-4e5d-8ee1-2adbe3b2e122 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:12.854727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497909a9-6874-4e5d-8ee1-2adbe3b2e122 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:12.854727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497909a9-6874-4e5d-8ee1-2adbe3b2e122 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:12.854727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 686/1350] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 20:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:12.861855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cecda-bd29-4d71-98e2-1ab6930df396 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:12.861855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cecda-bd29-4d71-98e2-1ab6930df396 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:12.861855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cecda-bd29-4d71-98e2-1ab6930df396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:12.966770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2e5cecda-bd29-4d71-98e2-1ab6930df396 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:12.967887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2e5cecda-bd29-4d71-98e2-1ab6930df396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:12.988078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:13.100926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 665/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:17.291957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efcdb3-ec2c-4f46-bf3f-d7b117134a2d None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:17.292749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efcdb3-ec2c-4f46-bf3f-d7b117134a2d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:17.293178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efcdb3-ec2c-4f46-bf3f-d7b117134a2d None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:17.295097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 687/1352] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 20:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:17.300012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58388a-8998-40c4-8837-fc62fa5ad456 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:17.300433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58388a-8998-40c4-8837-fc62fa5ad456 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:17.300674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58388a-8998-40c4-8837-fc62fa5ad456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:17.384717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ec58388a-8998-40c4-8837-fc62fa5ad456 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:17.385927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ec58388a-8998-40c4-8837-fc62fa5ad456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:17.463882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:17.504794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 666/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 20:57:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:18.700822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d72af-eb21-429c-8096-2e12bf9f4bab None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:18.700822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d72af-eb21-429c-8096-2e12bf9f4bab None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:18.701327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d72af-eb21-429c-8096-2e12bf9f4bab None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:18.702512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 688/1354] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 20:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:18.707488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc6a9-b0ea-4476-a326-5d4f85ed7f28 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:18.708335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc6a9-b0ea-4476-a326-5d4f85ed7f28 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:18.708620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc6a9-b0ea-4476-a326-5d4f85ed7f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:18.783540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ecafc6a9-b0ea-4476-a326-5d4f85ed7f28 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:18.784763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ecafc6a9-b0ea-4476-a326-5d4f85ed7f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:18.889587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 667/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 20:57:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:22.242356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be424a01-303a-49f6-9f64-5e3202828d4f None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:22.243296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be424a01-303a-49f6-9f64-5e3202828d4f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:22.243782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be424a01-303a-49f6-9f64-5e3202828d4f None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:22.246827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 689/1356] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:22.252836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca0de4-290b-4dfd-9cc9-c8ef12493782 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:22.253663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca0de4-290b-4dfd-9cc9-c8ef12493782 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:22.253944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca0de4-290b-4dfd-9cc9-c8ef12493782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:22.300018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:22.334405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b3ca0de4-290b-4dfd-9cc9-c8ef12493782 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:22.336331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b3ca0de4-290b-4dfd-9cc9-c8ef12493782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:22.438962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 668/1357] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:22.492119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3633259-cb4b-4100-b5ae-1a48bc4c12a0 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:22.492764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3633259-cb4b-4100-b5ae-1a48bc4c12a0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:22.493497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3633259-cb4b-4100-b5ae-1a48bc4c12a0 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:22.495990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 690/1358] 10.222.0.39 () {54 vars in 961 bytes} [Wed Jul 13 20:57:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:22.503415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488a41c-72f6-40bd-b62e-fb5b64ede760 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:22.504219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488a41c-72f6-40bd-b62e-fb5b64ede760 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:22.504966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488a41c-72f6-40bd-b62e-fb5b64ede760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:22.585514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8488a41c-72f6-40bd-b62e-fb5b64ede760 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:22.586749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8488a41c-72f6-40bd-b62e-fb5b64ede760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:22.634862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:22.710720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 669/1359] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Jul 13 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:57:22.711701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3633259-cb4b-4100-b5ae-1a48bc4c12a0 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:22.713210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d3633259-cb4b-4100-b5ae-1a48bc4c12a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:22.860028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3633259-cb4b-4100-b5ae-1a48bc4c12a0 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:22.934018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:22.976325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:22.977042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:22.977423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:22.977801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:22.979539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:23.078208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:23.079996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787ae44-1984-438f-93dc-a08741b964d2 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:23.099697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 691/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:57:25.345595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a613ce4-1c7d-4242-8e36-dc3991b6edf7 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:25.346141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a613ce4-1c7d-4242-8e36-dc3991b6edf7 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:25.346141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a613ce4-1c7d-4242-8e36-dc3991b6edf7 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:25.347744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 670/1361] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:25.354547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e8d4-619e-4af5-85a7-1c2b4688dc58 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:25.355099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e8d4-619e-4af5-85a7-1c2b4688dc58 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:25.355099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e8d4-619e-4af5-85a7-1c2b4688dc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:25.427216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-17e8e8d4-619e-4af5-85a7-1c2b4688dc58 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:25.429152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-17e8e8d4-619e-4af5-85a7-1c2b4688dc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:25.468611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:25.532061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 692/1362] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:25.576811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a613ce4-1c7d-4242-8e36-dc3991b6edf7 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:25.579266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7a613ce4-1c7d-4242-8e36-dc3991b6edf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:25.752920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a613ce4-1c7d-4242-8e36-dc3991b6edf7 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:25.762912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:25.896583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:25.896583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:25.897595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:25.897595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:25.899211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:26.003667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:26.005389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b73f70-ab7a-407d-ab89-7a41da8534ab service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:26.024423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 671/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:28.379387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cd22-a940-422b-a565-94fc04f476df None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:28.379907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cd22-a940-422b-a565-94fc04f476df None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:28.379991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cd22-a940-422b-a565-94fc04f476df None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:28.381297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 693/1364] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:28.387209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c392414-fa42-4321-afde-fb7dafed5c88 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:28.387657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c392414-fa42-4321-afde-fb7dafed5c88 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:28.387837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c392414-fa42-4321-afde-fb7dafed5c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:28.403371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:28.459285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4c392414-fa42-4321-afde-fb7dafed5c88 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:28.460841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4c392414-fa42-4321-afde-fb7dafed5c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:28.574630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 672/1365] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:28.620943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd77de-0ebf-4390-bec0-389aa308d8a8 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:28.621283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd77de-0ebf-4390-bec0-389aa308d8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:28.621594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd77de-0ebf-4390-bec0-389aa308d8a8 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:28.623403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 694/1366] 10.222.0.39 () {54 vars in 961 bytes} [Wed Jul 13 20:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:28.629267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297874b1-8ab7-4f39-a91d-c6f854622fd8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:28.629557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297874b1-8ab7-4f39-a91d-c6f854622fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:28.629836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297874b1-8ab7-4f39-a91d-c6f854622fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:28.709278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-297874b1-8ab7-4f39-a91d-c6f854622fd8 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:28.710520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-297874b1-8ab7-4f39-a91d-c6f854622fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:28.747387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:28.832974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 673/1367] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Jul 13 20:57:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:28.842662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd77de-0ebf-4390-bec0-389aa308d8a8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:28.844439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f1dd77de-0ebf-4390-bec0-389aa308d8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:29.013121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd77de-0ebf-4390-bec0-389aa308d8a8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:29.036106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:29.142115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:29.142797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:29.143083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:29.143358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:29.145002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:29.240101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:29.241751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876444ef-046b-4a7e-957d-ec0088decb83 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:29.260432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 695/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:30.316746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9248b-9539-4f33-85f0-f5a8b8832d44 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:30.317291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9248b-9539-4f33-85f0-f5a8b8832d44 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:30.317291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9248b-9539-4f33-85f0-f5a8b8832d44 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:30.318887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 674/1369] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:30.323912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb0189-fc03-4912-a0e0-962b960574fb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:30.324313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb0189-fc03-4912-a0e0-962b960574fb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:30.324531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb0189-fc03-4912-a0e0-962b960574fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:30.356032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:30.393651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4ccb0189-fc03-4912-a0e0-962b960574fb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:30.394892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4ccb0189-fc03-4912-a0e0-962b960574fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:30.497470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 696/1370] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:30.838809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adadbb5-eafe-4263-b786-fb403321cc50 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:30.839235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adadbb5-eafe-4263-b786-fb403321cc50 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:30.839651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adadbb5-eafe-4263-b786-fb403321cc50 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:30.841717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 675/1371] 10.222.0.39 () {54 vars in 963 bytes} [Wed Jul 13 20:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:30.846707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f97438-e54c-47cf-b9c5-5c3c5471b3e6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:30.847004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f97438-e54c-47cf-b9c5-5c3c5471b3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:30.847286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f97438-e54c-47cf-b9c5-5c3c5471b3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:30.927693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-61f97438-e54c-47cf-b9c5-5c3c5471b3e6 None None] MFA Rules not processed for user `9a58cb03e518413ab12cc84d793c4736`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:30.928974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-61f97438-e54c-47cf-b9c5-5c3c5471b3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:30.979759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:31.012569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 697/1372] 10.222.0.39 () {58 vars in 1060 bytes} [Wed Jul 13 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:31.019982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adadbb5-eafe-4263-b786-fb403321cc50 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:31.021750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8adadbb5-eafe-4263-b786-fb403321cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:31.144636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adadbb5-eafe-4263-b786-fb403321cc50 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:31.189645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:31.228215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:31.228885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:31.229179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:31.229508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:31.231469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:31.355855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:31.357302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf83fb-c87c-4ef9-ae31-3c33a3789945 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:31.375374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618dcd-049b-4b8c-ba0e-b34fe685658a None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:31.375870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618dcd-049b-4b8c-ba0e-b34fe685658a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:31.376009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24618dcd-049b-4b8c-ba0e-b34fe685658a None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:31.376900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 676/1373] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:57:31.377293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 698/1374] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:31.383118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c643b1-44b4-492d-9455-1145ce64e6c8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:31.383482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c643b1-44b4-492d-9455-1145ce64e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:31.383828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c643b1-44b4-492d-9455-1145ce64e6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:31.455948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-63c643b1-44b4-492d-9455-1145ce64e6c8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:31.457043 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-63c643b1-44b4-492d-9455-1145ce64e6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:31.480430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:31.566758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 677/1375] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:57:31.608231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618dcd-049b-4b8c-ba0e-b34fe685658a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:31.609782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-24618dcd-049b-4b8c-ba0e-b34fe685658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:31.802237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24618dcd-049b-4b8c-ba0e-b34fe685658a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:31.853855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:31.916057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:31.916704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:31.916984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:31.917257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:31.918934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:32.005808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:32.007502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018041a-008e-48bb-b548-27521d7b7906 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:32.026406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 699/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:34.076633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed363-cd33-41dc-82ca-12d5dec948d9 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:34.077143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed363-cd33-41dc-82ca-12d5dec948d9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:34.077143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ed363-cd33-41dc-82ca-12d5dec948d9 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:34.078486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 678/1377] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:34.083707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e418c-d26a-48e1-aad3-4595085b992d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:34.084113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e418c-d26a-48e1-aad3-4595085b992d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:34.084390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e418c-d26a-48e1-aad3-4595085b992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:34.149661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6c7e418c-d26a-48e1-aad3-4595085b992d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:34.150886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6c7e418c-d26a-48e1-aad3-4595085b992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:34.157667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:34.254471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 700/1378] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:34.304985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed363-cd33-41dc-82ca-12d5dec948d9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:34.306890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-321ed363-cd33-41dc-82ca-12d5dec948d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:34.466775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ed363-cd33-41dc-82ca-12d5dec948d9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:34.470489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:34.581526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:34.582424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:34.582741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:34.583065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:34.584707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:34.690932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:34.692536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87461e62-b9d3-4324-8d91-29eaa67f7af0 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:34.711342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 679/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:36.807475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b33d8-f8a5-4ce4-9c4a-5d051f7e1840 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:36.808050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b33d8-f8a5-4ce4-9c4a-5d051f7e1840 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:36.808050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b33d8-f8a5-4ce4-9c4a-5d051f7e1840 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:36.809655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 701/1380] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:36.815478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081fed2-7fb7-4a2d-9f41-a5203b609ce4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:36.816038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081fed2-7fb7-4a2d-9f41-a5203b609ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:36.816038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081fed2-7fb7-4a2d-9f41-a5203b609ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:36.827749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:36.892552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c081fed2-7fb7-4a2d-9f41-a5203b609ce4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:36.893694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c081fed2-7fb7-4a2d-9f41-a5203b609ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:36.996129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 680/1381] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:37.039855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b33d8-f8a5-4ce4-9c4a-5d051f7e1840 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:37.041431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d40b33d8-f8a5-4ce4-9c4a-5d051f7e1840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:37.197842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:37.209420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b33d8-f8a5-4ce4-9c4a-5d051f7e1840 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:37.335353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:37.336291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:37.336727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:37.337168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:37.339145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:37.433734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:37.435619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0f447-0a83-42b2-abfd-69144c445a2f service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:37.447177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:37.457046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 702/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:39.731287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033dc9c-eee9-425e-968d-bf3ae81c780c None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:39.731779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033dc9c-eee9-425e-968d-bf3ae81c780c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:39.731862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033dc9c-eee9-425e-968d-bf3ae81c780c None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:39.733251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 681/1383] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:39.739769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547eac-5e0d-4b4f-9734-ac663ec56903 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:39.740184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547eac-5e0d-4b4f-9734-ac663ec56903 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:39.740533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea547eac-5e0d-4b4f-9734-ac663ec56903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:39.819998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ea547eac-5e0d-4b4f-9734-ac663ec56903 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:39.821228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ea547eac-5e0d-4b4f-9734-ac663ec56903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:39.919330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 703/1384] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:39.960738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033dc9c-eee9-425e-968d-bf3ae81c780c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:39.962273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b033dc9c-eee9-425e-968d-bf3ae81c780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:40.050546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:40.113492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033dc9c-eee9-425e-968d-bf3ae81c780c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:40.229406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:40.230084 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:40.230387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:40.230700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:40.232408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:40.285997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:40.449362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:40.450998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8551803f-08ee-4083-a4d6-e1767e7b1cc4 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:40.468824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 682/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:42.548081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57844f95-f0a3-4e06-b16e-f4ebcf3c1c11 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:42.548782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57844f95-f0a3-4e06-b16e-f4ebcf3c1c11 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:42.548782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57844f95-f0a3-4e06-b16e-f4ebcf3c1c11 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:42.550015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 704/1386] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:42.555414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc04c7-3f30-434d-b402-1614cdb80e32 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:42.555885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc04c7-3f30-434d-b402-1614cdb80e32 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:42.556037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc04c7-3f30-434d-b402-1614cdb80e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:42.620997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-08cc04c7-3f30-434d-b402-1614cdb80e32 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:42.622231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-08cc04c7-3f30-434d-b402-1614cdb80e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:42.719597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 683/1387] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:42.764029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57844f95-f0a3-4e06-b16e-f4ebcf3c1c11 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:42.766370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-57844f95-f0a3-4e06-b16e-f4ebcf3c1c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:42.852357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:42.927476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57844f95-f0a3-4e06-b16e-f4ebcf3c1c11 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:43.050910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:43.051796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:43.052151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:43.052506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:43.054833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:43.100028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:43.154929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:43.156606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3b09a-8bd0-40e2-8d75-73f2ff934b57 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:43.174910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 705/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:45.287802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c91e-15e6-4b39-a160-96a587ec4b02 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:45.288810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c91e-15e6-4b39-a160-96a587ec4b02 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:45.288924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c91e-15e6-4b39-a160-96a587ec4b02 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:45.290240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 684/1389] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:45.295288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1426-3aa9-4a2c-9942-2e0344d44826 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:45.295735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1426-3aa9-4a2c-9942-2e0344d44826 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:45.295826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1426-3aa9-4a2c-9942-2e0344d44826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:45.359007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-82ee1426-3aa9-4a2c-9942-2e0344d44826 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:45.360187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-82ee1426-3aa9-4a2c-9942-2e0344d44826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:45.457197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 706/1390] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:45.500953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6c91e-15e6-4b39-a160-96a587ec4b02 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:45.502523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-16d6c91e-15e6-4b39-a160-96a587ec4b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:45.534029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:45.648390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6c91e-15e6-4b39-a160-96a587ec4b02 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:45.749298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:45.761906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:45.762559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:45.762883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:45.763162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:45.764808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:45.854968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:45.856485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe743a-8657-40bc-8514-2c9beb803f03 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:45.874710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 685/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:46.352089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3093bf6-8bb4-4de3-a1b5-b8830778fbc1 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:46.352657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3093bf6-8bb4-4de3-a1b5-b8830778fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:46.352761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3093bf6-8bb4-4de3-a1b5-b8830778fbc1 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:46.354163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 707/1392] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:46.359348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77cf9-ef77-406a-a9ce-8db1c620c59c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:46.359756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77cf9-ef77-406a-a9ce-8db1c620c59c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:46.359916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77cf9-ef77-406a-a9ce-8db1c620c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:46.423674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-84b77cf9-ef77-406a-a9ce-8db1c620c59c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:46.424883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-84b77cf9-ef77-406a-a9ce-8db1c620c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:46.483254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:46.522113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 686/1393] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:46.577870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55595574-780f-453d-9730-acb0d5a3a11e None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:46.578157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55595574-780f-453d-9730-acb0d5a3a11e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:46.578434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55595574-780f-453d-9730-acb0d5a3a11e None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:46.579834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 708/1394] 10.222.0.39 () {54 vars in 962 bytes} [Wed Jul 13 20:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:46.584787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca031-d1d6-4a22-be8c-f5068b77444b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:46.585179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca031-d1d6-4a22-be8c-f5068b77444b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:46.585540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca031-d1d6-4a22-be8c-f5068b77444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:46.661430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-616ca031-d1d6-4a22-be8c-f5068b77444b None None] MFA Rules not processed for user `a6f4dc9555784337b2fd930caea78fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:46.662658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-616ca031-d1d6-4a22-be8c-f5068b77444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:46.740881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 687/1395] 10.222.0.39 () {58 vars in 1059 bytes} [Wed Jul 13 20:57:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:46.749118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55595574-780f-453d-9730-acb0d5a3a11e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:46.750783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-55595574-780f-453d-9730-acb0d5a3a11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:46.764548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:46.857797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55595574-780f-453d-9730-acb0d5a3a11e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:46.933062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:46.933720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:46.933995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:46.934278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:46.935947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:47.007512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:47.051237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:47.052667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5ac6d-61ab-4a44-8759-9f59adc47428 service glance] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:47.070466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 709/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:47.767495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c1e06-ab5d-4033-8d31-64759603ea99 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:47.767961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c1e06-ab5d-4033-8d31-64759603ea99 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:47.768046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c1e06-ab5d-4033-8d31-64759603ea99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:47.834735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2c5c1e06-ab5d-4033-8d31-64759603ea99 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:47.835876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2c5c1e06-ab5d-4033-8d31-64759603ea99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:47.844735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:47.925833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 688/1397] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:47.935328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d0d28-6067-4eb2-893c-5b2a121d9746 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:47.936050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d0d28-6067-4eb2-893c-5b2a121d9746 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:47.936050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d0d28-6067-4eb2-893c-5b2a121d9746 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:47.937045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 710/1398] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:47.942460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:47.942804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:47.943842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:47.973156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548450e-b644-42b0-8876-64034978f412 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:47.973444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548450e-b644-42b0-8876-64034978f412 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:47.973725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548450e-b644-42b0-8876-64034978f412 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:47.975131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 711/1399] 10.222.0.39 () {54 vars in 942 bytes} [Wed Jul 13 20:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:57:47.981108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:47.981405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:47.981684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:48.013026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:48.014226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:48.057940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] MFA Rules not processed for user `292fed9f5de945c883cc1ffeb91da4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:48.059178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:48.111037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 689/1400] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:48.154914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:48.156259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:48.158847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:48.167821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 712/1401] 10.222.0.39 () {58 vars in 1039 bytes} [Wed Jul 13 20:57:47 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:57:48.177083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:48.182319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:48.188254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2570a2-eed8-48af-9ad2-963e8499b553 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:48.276797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9baf2-3b62-406b-8cf3-20ae5c023ea8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:48.293677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:48.294327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:48.294609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:48.294915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:48.296619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:48.381870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:48.383423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08143ad0-c582-4193-88c2-f81d90d5e04b service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:48.389998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:48.392502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 713/1402] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 20:57:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 20:57:48.393250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:48.394175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:48.394696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:48.394978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:48.396888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:48.497078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:48.498796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f918e4d-0cfe-40a0-9953-4af6bfa10374 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:48.524377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 690/1403] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 20:57:50.610585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8263b21-7220-44a2-bdf5-656198d2fa9c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:50.611137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8263b21-7220-44a2-bdf5-656198d2fa9c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:50.611223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8263b21-7220-44a2-bdf5-656198d2fa9c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:50.612701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 714/1404] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:50.618292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784d1b0-fb32-4aca-9c2b-0840ff04c877 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:50.618758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784d1b0-fb32-4aca-9c2b-0840ff04c877 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:50.618879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784d1b0-fb32-4aca-9c2b-0840ff04c877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:50.688338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b784d1b0-fb32-4aca-9c2b-0840ff04c877 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:50.689636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b784d1b0-fb32-4aca-9c2b-0840ff04c877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:50.774525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:50.799686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 691/1405] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:50.844077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8263b21-7220-44a2-bdf5-656198d2fa9c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:50.845631 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c8263b21-7220-44a2-bdf5-656198d2fa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:50.881793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:51.024169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8263b21-7220-44a2-bdf5-656198d2fa9c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:51.152471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:51.165996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:51.166687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:51.166969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:51.167240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:51.168869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:51.271258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:51.273177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5165b-95eb-4af8-bfe8-04f50a6e84b7 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:51.293074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 715/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:57:53.330206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a55fe6-afe9-43ca-a230-a9a5685c6711 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:53.330775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a55fe6-afe9-43ca-a230-a9a5685c6711 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:53.330775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a55fe6-afe9-43ca-a230-a9a5685c6711 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:53.332153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 692/1407] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:57:53.337734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b728b4-d3d2-48b2-9924-14792720418a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:53.338175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b728b4-d3d2-48b2-9924-14792720418a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:53.338288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b728b4-d3d2-48b2-9924-14792720418a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:53.405959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f6b728b4-d3d2-48b2-9924-14792720418a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:57:53.407212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f6b728b4-d3d2-48b2-9924-14792720418a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:53.473910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:53.521568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 716/1408] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:57:53.566156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a55fe6-afe9-43ca-a230-a9a5685c6711 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:57:53.567729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e9a55fe6-afe9-43ca-a230-a9a5685c6711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:53.730134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a55fe6-afe9-43ca-a230-a9a5685c6711 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:57:53.790159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:57:53.856974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:57:53.857687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:57:53.858037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:57:53.858318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:57:53.859998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:57:53.961897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:57:53.963619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b57717-52ff-4ed5-b05b-e4daebb59bfb service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:57:53.982590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 693/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:03.517693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=14 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:03.980055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=73 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:10.223894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6127f43-0cba-4758-a105-9d8207080fde None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:10.224470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6127f43-0cba-4758-a105-9d8207080fde None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:10.224470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6127f43-0cba-4758-a105-9d8207080fde None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:10.225855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 717/1410] 10.222.0.39 () {54 vars in 918 bytes} [Wed Jul 13 20:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:10.231091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c9a8-bf01-4133-8460-84d869de3786 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:10.231542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c9a8-bf01-4133-8460-84d869de3786 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:10.231679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c9a8-bf01-4133-8460-84d869de3786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:10.305792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1399c9a8-bf01-4133-8460-84d869de3786 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:10.307062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1399c9a8-bf01-4133-8460-84d869de3786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:10.410678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 694/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Jul 13 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:10.424501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c12b0-6426-403c-b79f-63181a5067ad None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:10.424812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c12b0-6426-403c-b79f-63181a5067ad None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:10.425090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c12b0-6426-403c-b79f-63181a5067ad None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:10.426434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 718/1412] 10.222.0.39 () {54 vars in 918 bytes} [Wed Jul 13 20:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:10.431573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0393-4d91-4a61-b9db-867e76847418 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:10.431855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0393-4d91-4a61-b9db-867e76847418 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:10.432127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0393-4d91-4a61-b9db-867e76847418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:10.500787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:10.503234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-458c0393-4d91-4a61-b9db-867e76847418 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:10.504940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-458c0393-4d91-4a61-b9db-867e76847418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:10.619440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 695/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Wed Jul 13 20:58:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:13.100021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd33c-41d0-4346-8061-fd613496316b None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:13.100552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd33c-41d0-4346-8061-fd613496316b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:13.100552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fd33c-41d0-4346-8061-fd613496316b None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:13.102415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:13.107593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c8f9f-994b-4535-9ab6-e49f2c17040c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:13.107991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c8f9f-994b-4535-9ab6-e49f2c17040c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:13.108160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c8f9f-994b-4535-9ab6-e49f2c17040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:13.172581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-309c8f9f-994b-4535-9ab6-e49f2c17040c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:13.173792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-309c8f9f-994b-4535-9ab6-e49f2c17040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:13.228752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:13.269733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:13.313373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fd33c-41d0-4346-8061-fd613496316b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:13.314932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9e3fd33c-41d0-4346-8061-fd613496316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:13.456983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fd33c-41d0-4346-8061-fd613496316b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:13.527152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:13.567506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:13.568143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:13.568422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:13.568691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:13.570308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:13.659245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:13.660840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797017-9cc4-42e0-a7e7-4da22b462b60 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:13.678535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 720/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:14.568991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025976-1224-45a7-bcad-8febfccab25b None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:14.569505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025976-1224-45a7-bcad-8febfccab25b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:14.569505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025976-1224-45a7-bcad-8febfccab25b None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:14.570906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:14.575678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13b8d7-6585-4032-93f4-b814625960f6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:14.576061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13b8d7-6585-4032-93f4-b814625960f6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:14.576241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13b8d7-6585-4032-93f4-b814625960f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:14.648207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0f13b8d7-6585-4032-93f4-b814625960f6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:14.649393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0f13b8d7-6585-4032-93f4-b814625960f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:14.676223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:14.745480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:14.894180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9025976-1224-45a7-bcad-8febfccab25b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:14.895736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b9025976-1224-45a7-bcad-8febfccab25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:15.037784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9025976-1224-45a7-bcad-8febfccab25b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:15.077434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:15.150884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:15.151457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:15.151738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:15.152011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:15.153632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:15.239196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:15.240756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc9297-b505-4cd5-8a68-f7e9fdf18893 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:15.259104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 698/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:16.127889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2902-d7e7-49b1-82ed-41c8c2d0f938 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:16.128468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2902-d7e7-49b1-82ed-41c8c2d0f938 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:16.128468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc2902-d7e7-49b1-82ed-41c8c2d0f938 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:16.129793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:16.134672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5f61e-fafb-4d7e-91a5-305a1bf0c7e2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:16.135105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5f61e-fafb-4d7e-91a5-305a1bf0c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:16.135217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5f61e-fafb-4d7e-91a5-305a1bf0c7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:16.188827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:16.210323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fbb5f61e-fafb-4d7e-91a5-305a1bf0c7e2 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:16.211575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fbb5f61e-fafb-4d7e-91a5-305a1bf0c7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:16.308661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:58:16.455929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc2902-d7e7-49b1-82ed-41c8c2d0f938 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:16.457358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-debc2902-d7e7-49b1-82ed-41c8c2d0f938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:16.598310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc2902-d7e7-49b1-82ed-41c8c2d0f938 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:16.609981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:16.709094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:16.709663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:16.709942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:16.710210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:16.711888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:16.798836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:16.800409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22918a19-c2cc-4a9d-9faf-3186c2b5eb46 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:16.817770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 723/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:17.668192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041bd21-ec90-4445-af01-c5fbee7917cb None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:17.668726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041bd21-ec90-4445-af01-c5fbee7917cb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:17.668816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041bd21-ec90-4445-af01-c5fbee7917cb None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:17.670142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:17.674978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001a7c-7d0e-48ae-97f7-e2ddeba4ecdb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:17.675440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001a7c-7d0e-48ae-97f7-e2ddeba4ecdb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:17.675548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31001a7c-7d0e-48ae-97f7-e2ddeba4ecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:17.694502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:17.750465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-31001a7c-7d0e-48ae-97f7-e2ddeba4ecdb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:17.751689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-31001a7c-7d0e-48ae-97f7-e2ddeba4ecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:17.846531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:17.994205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041bd21-ec90-4445-af01-c5fbee7917cb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:17.995667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4041bd21-ec90-4445-af01-c5fbee7917cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:18.106866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:18.142396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041bd21-ec90-4445-af01-c5fbee7917cb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:18.252567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:18.253144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:18.253431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:18.253706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:18.255448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:18.330745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:18.344203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:18.345791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed13ef3-3d7f-43b7-b453-54ac55b4e576 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:18.363802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 701/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:19.215603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dd5b5-384d-497e-b929-bbd45e529e50 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:19.216165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dd5b5-384d-497e-b929-bbd45e529e50 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:19.216165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7dd5b5-384d-497e-b929-bbd45e529e50 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:19.217575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:19.222379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0437bc-73e1-403a-ae30-4c173f073858 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:19.222786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0437bc-73e1-403a-ae30-4c173f073858 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:19.222971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0437bc-73e1-403a-ae30-4c173f073858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:19.297997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4b0437bc-73e1-403a-ae30-4c173f073858 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:19.299218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4b0437bc-73e1-403a-ae30-4c173f073858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:19.396110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 20:58:19.545216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dd5b5-384d-497e-b929-bbd45e529e50 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:19.546746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fe7dd5b5-384d-497e-b929-bbd45e529e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:19.629094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:19.689752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7dd5b5-384d-497e-b929-bbd45e529e50 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:19.800062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:19.800636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:19.800919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:19.801201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:19.802867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:19.849894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:19.891540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:19.893145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0df91-fc96-4f2c-8c0f-5d840c98fff5 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:19.911017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 726/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:20.700982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424642d-c74a-45e3-848b-fe6464d54492 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:20.701546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424642d-c74a-45e3-848b-fe6464d54492 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:20.701546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a424642d-c74a-45e3-848b-fe6464d54492 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:20.702960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:20.707737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0e4f-8dc3-4846-a52a-7b912bbe9b53 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:20.708151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0e4f-8dc3-4846-a52a-7b912bbe9b53 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:20.708326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a0e4f-8dc3-4846-a52a-7b912bbe9b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:20.782056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e39a0e4f-8dc3-4846-a52a-7b912bbe9b53 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:20.783279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e39a0e4f-8dc3-4846-a52a-7b912bbe9b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:20.876777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:21.024114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424642d-c74a-45e3-848b-fe6464d54492 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:21.025563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a424642d-c74a-45e3-848b-fe6464d54492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:21.074223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:21.169551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a424642d-c74a-45e3-848b-fe6464d54492 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:21.280362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:21.281023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:21.281305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:21.281579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:21.283253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:21.290204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:21.372430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:21.374023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c142486-4e95-4fb2-9cac-32bc6afbe511 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:21.391742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 704/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:22.185812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a9fc-411d-4dfe-a4ab-b7911d582dba None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:22.186406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a9fc-411d-4dfe-a4ab-b7911d582dba None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:22.186406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792a9fc-411d-4dfe-a4ab-b7911d582dba None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:22.187768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:22.192769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d00bc3-13a7-4232-8cd6-4dc40d124de6 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:22.193170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d00bc3-13a7-4232-8cd6-4dc40d124de6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:22.193362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d00bc3-13a7-4232-8cd6-4dc40d124de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:22.268209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-88d00bc3-13a7-4232-8cd6-4dc40d124de6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:22.269571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-88d00bc3-13a7-4232-8cd6-4dc40d124de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:22.336607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:22.365992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:22.515266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a9fc-411d-4dfe-a4ab-b7911d582dba None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:22.516698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c792a9fc-411d-4dfe-a4ab-b7911d582dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:22.534887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:22.660614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792a9fc-411d-4dfe-a4ab-b7911d582dba None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:22.746160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:22.772094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:22.772677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:22.772960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:22.773233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:22.775500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:22.861936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:22.863551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfd952-fc7c-4226-8ddb-f1d2c9e5374e service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:22.881248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 729/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:23.678642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47014c3-37ac-42b6-9561-c32178f69bc4 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:23.679173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47014c3-37ac-42b6-9561-c32178f69bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:23.679290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47014c3-37ac-42b6-9561-c32178f69bc4 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:23.680593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:23.686085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591fef-3ffa-4db0-b9b2-6c4f05bb660c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:23.686504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591fef-3ffa-4db0-b9b2-6c4f05bb660c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:23.686644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f591fef-3ffa-4db0-b9b2-6c4f05bb660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:23.759678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-3f591fef-3ffa-4db0-b9b2-6c4f05bb660c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:23.760922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3f591fef-3ffa-4db0-b9b2-6c4f05bb660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:23.802486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:23.856799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:24.004588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47014c3-37ac-42b6-9561-c32178f69bc4 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:24.006035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c47014c3-37ac-42b6-9561-c32178f69bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:24.156199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47014c3-37ac-42b6-9561-c32178f69bc4 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:24.208549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:24.271232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:24.271814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:24.272103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:24.272402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:24.274046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:24.362929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:24.364515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c52cd0-5b96-4d49-8658-a927dad98a7f service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:24.381918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 707/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:25.178595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062030d5-cff3-4bc2-9c7a-9d42bd721d9d None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:25.179183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062030d5-cff3-4bc2-9c7a-9d42bd721d9d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:25.179183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062030d5-cff3-4bc2-9c7a-9d42bd721d9d None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:25.180548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 20:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:25.186167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954a10c-19c1-4d52-a093-e412ce57d03f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:25.186572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954a10c-19c1-4d52-a093-e412ce57d03f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:25.186751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954a10c-19c1-4d52-a093-e412ce57d03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:25.260925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d954a10c-19c1-4d52-a093-e412ce57d03f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:25.262127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d954a10c-19c1-4d52-a093-e412ce57d03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:25.264159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:25.358272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 20:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:25.505425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062030d5-cff3-4bc2-9c7a-9d42bd721d9d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:25.506885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-062030d5-cff3-4bc2-9c7a-9d42bd721d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:25.649528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062030d5-cff3-4bc2-9c7a-9d42bd721d9d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:25.676530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:25.762481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:25.763077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:25.763360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:25.763637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:25.765295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:25.852480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:25.854126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270f2d3-63bd-44ed-8d07-1021af610584 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:25.871821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 732/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:27.780719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0f24-241f-480b-8e33-befb3c62dc17 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:27.781250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0f24-241f-480b-8e33-befb3c62dc17 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:27.781250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0f24-241f-480b-8e33-befb3c62dc17 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:27.782676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:27.787784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62961c67-057b-4cf7-8b0a-6243d8f63984 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:27.788194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62961c67-057b-4cf7-8b0a-6243d8f63984 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:27.788339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62961c67-057b-4cf7-8b0a-6243d8f63984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:27.828113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:27.854061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-62961c67-057b-4cf7-8b0a-6243d8f63984 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:27.855289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-62961c67-057b-4cf7-8b0a-6243d8f63984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:27.949725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:27.988165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff0f24-241f-480b-8e33-befb3c62dc17 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:27.989707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6fff0f24-241f-480b-8e33-befb3c62dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:28.093723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff0f24-241f-480b-8e33-befb3c62dc17 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:28.134891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:28.177461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:28.178112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:28.178395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:28.178718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:28.180372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:28.287564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:28.288999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2629fa-9e4d-4bfe-9b9b-0e3728d60b30 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:28.307642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 710/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:30.463285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732e369-b470-4d25-a0f8-16657bbc6250 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:30.463840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732e369-b470-4d25-a0f8-16657bbc6250 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:30.463840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732e369-b470-4d25-a0f8-16657bbc6250 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:30.465204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:30.470479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893574a-ab88-45cf-9dca-1d34e00539c4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:30.470877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893574a-ab88-45cf-9dca-1d34e00539c4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:30.471046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893574a-ab88-45cf-9dca-1d34e00539c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:30.534836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f893574a-ab88-45cf-9dca-1d34e00539c4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:30.535979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f893574a-ab88-45cf-9dca-1d34e00539c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:30.548960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:30.634207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:30.672024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732e369-b470-4d25-a0f8-16657bbc6250 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:30.673548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d732e369-b470-4d25-a0f8-16657bbc6250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:30.776672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732e369-b470-4d25-a0f8-16657bbc6250 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:30.788905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:30.849973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:30.850647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:30.850949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:30.851230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:30.852903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:30.961137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:30.963390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9318e-ec82-4df4-8844-5ba547b699d9 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:30.981371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 735/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:33.323502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32b7a-482c-4836-94b2-493fd44bda5c None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:33.324015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32b7a-482c-4836-94b2-493fd44bda5c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:33.324132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32b7a-482c-4836-94b2-493fd44bda5c None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:33.325425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:33.330533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcec09-e40c-4b42-86ee-21d5d257b9cc None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:33.330977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcec09-e40c-4b42-86ee-21d5d257b9cc None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:33.331127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fcec09-e40c-4b42-86ee-21d5d257b9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:33.376915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:33.395024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-45fcec09-e40c-4b42-86ee-21d5d257b9cc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:33.396218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-45fcec09-e40c-4b42-86ee-21d5d257b9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:33.490159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:33.528278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32b7a-482c-4836-94b2-493fd44bda5c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:33.530021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-31e32b7a-482c-4836-94b2-493fd44bda5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:33.632262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32b7a-482c-4836-94b2-493fd44bda5c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:33.671886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:33.705301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:33.705946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:33.706235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:33.706512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:33.708203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:33.814686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:33.816301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc52fb1-1274-44b9-b639-2daa993d520d service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:33.834958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 713/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:35.971675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d88712-e057-48ed-8eab-da7d37d01998 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:35.972266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d88712-e057-48ed-8eab-da7d37d01998 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:35.972266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d88712-e057-48ed-8eab-da7d37d01998 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:35.973617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:35.978897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d674-b3f0-43a4-ae97-1193db35027b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:35.979353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d674-b3f0-43a4-ae97-1193db35027b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:35.979458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3d674-b3f0-43a4-ae97-1193db35027b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:36.045660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1fd3d674-b3f0-43a4-ae97-1193db35027b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:36.046896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1fd3d674-b3f0-43a4-ae97-1193db35027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:36.067806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:36.144097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:36.183743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d88712-e057-48ed-8eab-da7d37d01998 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:36.185338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c2d88712-e057-48ed-8eab-da7d37d01998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:36.286463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d88712-e057-48ed-8eab-da7d37d01998 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:36.307055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:36.359955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:36.360597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:36.360885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:36.361162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:36.362826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:36.469909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:36.471363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e27e8c-10ee-4eee-a527-ab5d269b633c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:36.489505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 738/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:38.667092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e01a-5502-4d9c-8284-bd979583f683 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:38.667658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e01a-5502-4d9c-8284-bd979583f683 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:38.667658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2e01a-5502-4d9c-8284-bd979583f683 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:38.669017 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:38.674281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502930-0113-44ae-87b9-9897c148d38c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:38.674728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502930-0113-44ae-87b9-9897c148d38c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:38.674889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60502930-0113-44ae-87b9-9897c148d38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:38.737530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:38.740455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-60502930-0113-44ae-87b9-9897c148d38c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:38.742245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-60502930-0113-44ae-87b9-9897c148d38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:38.835725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:38.873549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e01a-5502-4d9c-8284-bd979583f683 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:38.875120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0bf2e01a-5502-4d9c-8284-bd979583f683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:38.975557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2e01a-5502-4d9c-8284-bd979583f683 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:39.023217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:39.048634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:39.049283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:39.049566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:39.049842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:39.051508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:39.157864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:39.159342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a5b65-977f-424e-b0d6-72c728c344c8 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:39.177157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 716/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:41.292706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d672443-9f22-4d62-809c-247e72ed2866 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:41.293151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d672443-9f22-4d62-809c-247e72ed2866 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:41.293293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d672443-9f22-4d62-809c-247e72ed2866 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:41.294712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:41.299863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83dfd-89eb-4d1a-b02b-92e3763b4c8a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:41.300329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83dfd-89eb-4d1a-b02b-92e3763b4c8a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:41.300446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83dfd-89eb-4d1a-b02b-92e3763b4c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:41.363882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-93b83dfd-89eb-4d1a-b02b-92e3763b4c8a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:41.365082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-93b83dfd-89eb-4d1a-b02b-92e3763b4c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:41.394671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:41.461213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:41.499320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d672443-9f22-4d62-809c-247e72ed2866 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:41.500850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0d672443-9f22-4d62-809c-247e72ed2866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:41.601446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d672443-9f22-4d62-809c-247e72ed2866 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:41.629786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:41.674447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:41.675126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:41.675427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:41.675700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:41.677329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:41.783388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:41.784807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe9afb-8ebd-4903-974b-ca03fc8c183b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:41.802819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 741/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:43.862303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa6ec9-4b70-4d3a-b504-1ef48c51d80d None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:43.862875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa6ec9-4b70-4d3a-b504-1ef48c51d80d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:43.862997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa6ec9-4b70-4d3a-b504-1ef48c51d80d None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:43.864313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:43.869562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088a8c0-61dd-4ba4-a4a5-3ff474d3743b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:43.870008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088a8c0-61dd-4ba4-a4a5-3ff474d3743b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:43.870113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088a8c0-61dd-4ba4-a4a5-3ff474d3743b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:43.934453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-8088a8c0-61dd-4ba4-a4a5-3ff474d3743b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:43.935657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8088a8c0-61dd-4ba4-a4a5-3ff474d3743b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:43.943781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:44.033279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:44.069892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa6ec9-4b70-4d3a-b504-1ef48c51d80d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:44.071549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f1fa6ec9-4b70-4d3a-b504-1ef48c51d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:44.174070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa6ec9-4b70-4d3a-b504-1ef48c51d80d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:44.228378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:44.246817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:44.247476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:44.247758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:44.248033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:44.249676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:44.361484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:44.362949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a3f02-202f-447c-950f-e0e52079ca7a service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:44.381461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 719/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:46.432639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411545d-6af1-47a5-a4ec-5e0f0d23e3e7 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:46.433171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411545d-6af1-47a5-a4ec-5e0f0d23e3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:46.433289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411545d-6af1-47a5-a4ec-5e0f0d23e3e7 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:46.434638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:46.439896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcf57-869c-42ae-9b0d-62baa9a0e222 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:46.440338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcf57-869c-42ae-9b0d-62baa9a0e222 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:46.440480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcf57-869c-42ae-9b0d-62baa9a0e222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:46.504209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-caabcf57-869c-42ae-9b0d-62baa9a0e222 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:46.505407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-caabcf57-869c-42ae-9b0d-62baa9a0e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:46.543617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:46.601474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:46.638692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411545d-6af1-47a5-a4ec-5e0f0d23e3e7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:46.640241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f411545d-6af1-47a5-a4ec-5e0f0d23e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:46.741475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411545d-6af1-47a5-a4ec-5e0f0d23e3e7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:46.779272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:46.816010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:46.816593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:46.816878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:46.817155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:46.818814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:46.924450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:46.925872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a9c86-bd92-4d71-ab3f-b456cfbbe31c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:46.943870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 744/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:48.970745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811105f1-39fc-4b2f-b2fa-502d4a3fa25e None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:48.971298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811105f1-39fc-4b2f-b2fa-502d4a3fa25e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:48.971298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811105f1-39fc-4b2f-b2fa-502d4a3fa25e None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:48.972669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:48.977776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f8816-0284-4ede-b25f-fb815e527f4d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:48.978262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f8816-0284-4ede-b25f-fb815e527f4d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:48.978373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f8816-0284-4ede-b25f-fb815e527f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:49.041477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bd8f8816-0284-4ede-b25f-fb815e527f4d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:49.042597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bd8f8816-0284-4ede-b25f-fb815e527f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:49.059703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:49.136931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:49.172971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811105f1-39fc-4b2f-b2fa-502d4a3fa25e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:49.174524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-811105f1-39fc-4b2f-b2fa-502d4a3fa25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:49.276611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811105f1-39fc-4b2f-b2fa-502d4a3fa25e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:49.342280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:49.349232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:49.349870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:49.350152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:49.350424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:49.352092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:49.457745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:49.459215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d474e-b087-4cd4-9a8f-4cefa116dc29 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:49.476966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 722/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:58:51.502109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806bfe-486d-440c-bc5e-7d824b97d04c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:51.502681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806bfe-486d-440c-bc5e-7d824b97d04c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:51.502778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806bfe-486d-440c-bc5e-7d824b97d04c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:51.504104 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 20:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:58:51.509249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609178e2-cb26-4a05-a902-53abe5007d32 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:51.509663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609178e2-cb26-4a05-a902-53abe5007d32 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:51.509802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609178e2-cb26-4a05-a902-53abe5007d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:51.572698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-609178e2-cb26-4a05-a902-53abe5007d32 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:58:51.573818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-609178e2-cb26-4a05-a902-53abe5007d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:51.621271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:51.668283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 20:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:58:51.703362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1806bfe-486d-440c-bc5e-7d824b97d04c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:58:51.704891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f1806bfe-486d-440c-bc5e-7d824b97d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:51.808248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1806bfe-486d-440c-bc5e-7d824b97d04c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:58:51.855108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:58:51.881041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:58:51.881686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:58:51.881967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:58:51.882240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:58:51.883919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:58:51.989815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:58:51.991286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d21e-37e2-4791-996c-31ebc5946679 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:58:52.008748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 747/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:01.666481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=22 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:02.006531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=66 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:12.329847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07211f9b-81d2-48a6-85c1-6c36e616d535 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:12.330385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07211f9b-81d2-48a6-85c1-6c36e616d535 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:12.330385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07211f9b-81d2-48a6-85c1-6c36e616d535 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:12.331830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 724/1471] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:12.337964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3295ee1-9988-4e88-bb2a-2a2f4f969911 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:12.338388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3295ee1-9988-4e88-bb2a-2a2f4f969911 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:12.338573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3295ee1-9988-4e88-bb2a-2a2f4f969911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:12.401466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-d3295ee1-9988-4e88-bb2a-2a2f4f969911 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:12.402597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d3295ee1-9988-4e88-bb2a-2a2f4f969911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:12.496153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 748/1472] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:12.579181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239645ea-1624-45ce-ac56-de57c6baf9df None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:12.579558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239645ea-1624-45ce-ac56-de57c6baf9df None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:12.579842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239645ea-1624-45ce-ac56-de57c6baf9df None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:12.581222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 725/1473] 10.222.0.39 () {54 vars in 962 bytes} [Wed Jul 13 20:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 20:59:12.586097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c275a1-09e9-4ab7-82d3-c4af80ec90a5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:12.586385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c275a1-09e9-4ab7-82d3-c4af80ec90a5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:12.586695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c275a1-09e9-4ab7-82d3-c4af80ec90a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:12.659424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-72c275a1-09e9-4ab7-82d3-c4af80ec90a5 None None] MFA Rules not processed for user `a6f4dc9555784337b2fd930caea78fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:12.660554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-72c275a1-09e9-4ab7-82d3-c4af80ec90a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:12.680309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:12.735939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 749/1474] 10.222.0.39 () {58 vars in 1059 bytes} [Wed Jul 13 20:59:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:12.743746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239645ea-1624-45ce-ac56-de57c6baf9df None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:12.745257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-239645ea-1624-45ce-ac56-de57c6baf9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:12.847902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239645ea-1624-45ce-ac56-de57c6baf9df None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:12.919413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:12.919991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:12.920275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:12.920550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:12.922182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:12.974397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:13.034675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:13.036278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7496e1e-059f-4593-bc0d-dde2c5154255 service glance] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:13.055568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 726/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:15.133703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b04ce-a244-485c-bf5a-2269cb6cc6e1 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:15.134788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b04ce-a244-485c-bf5a-2269cb6cc6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:15.134788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b04ce-a244-485c-bf5a-2269cb6cc6e1 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:15.136155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 750/1476] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:15.141244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fabcf8-047f-46de-b0e5-595628c55481 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:15.141697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fabcf8-047f-46de-b0e5-595628c55481 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:15.141851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fabcf8-047f-46de-b0e5-595628c55481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:15.205311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-62fabcf8-047f-46de-b0e5-595628c55481 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:15.206490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-62fabcf8-047f-46de-b0e5-595628c55481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:15.301699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 727/1477] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:15.325283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b04ce-a244-485c-bf5a-2269cb6cc6e1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:15.326825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e34b04ce-a244-485c-bf5a-2269cb6cc6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:15.448233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b04ce-a244-485c-bf5a-2269cb6cc6e1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:15.494805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:15.538214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:15.538898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:15.539180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:15.539452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:15.540178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:15.541946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 20:59:15.541946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 20:59:15.541946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 20:59:15.543122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a000d4-e79d-4537-80d0-006ac15b8627 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:15.561871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 751/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Wed Jul 13 20:59:15 2022] GET /identity/v3/projects => generated 2553 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 20:59:17.570331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2092a-a94f-4756-9a7f-b79414678eba None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:17.570861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2092a-a94f-4756-9a7f-b79414678eba None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:17.571029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2092a-a94f-4756-9a7f-b79414678eba None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:17.572474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 728/1479] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:17.577525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51208316-d828-440f-b480-6089328e1498 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:17.577935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51208316-d828-440f-b480-6089328e1498 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:17.578080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51208316-d828-440f-b480-6089328e1498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:17.640729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-51208316-d828-440f-b480-6089328e1498 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:17.641917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-51208316-d828-440f-b480-6089328e1498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:17.736437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 752/1480] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:17.778358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2092a-a94f-4756-9a7f-b79414678eba None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:17.780178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9eb2092a-a94f-4756-9a7f-b79414678eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:17.789849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:17.924574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2092a-a94f-4756-9a7f-b79414678eba None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:18.001716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:18.034134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:18.034808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:18.035093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:18.035367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:18.037001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:18.123364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:18.124935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ea387-c3ed-4daa-ac4f-91274ea4aeca service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:18.143198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 729/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:20.343365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebd7ea-1c6f-4e9d-89cc-c95e42582681 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:20.343896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebd7ea-1c6f-4e9d-89cc-c95e42582681 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:20.343896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebd7ea-1c6f-4e9d-89cc-c95e42582681 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:20.345278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 753/1482] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:20.350700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c59b2-bc69-493f-9f7c-fa2bf945ecf9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:20.351108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c59b2-bc69-493f-9f7c-fa2bf945ecf9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:20.351259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c59b2-bc69-493f-9f7c-fa2bf945ecf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:20.415529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e15c59b2-bc69-493f-9f7c-fa2bf945ecf9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:20.416721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e15c59b2-bc69-493f-9f7c-fa2bf945ecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:20.461494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:20.511598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 730/1483] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:20.547152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebd7ea-1c6f-4e9d-89cc-c95e42582681 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:20.548681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-35ebd7ea-1c6f-4e9d-89cc-c95e42582681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:20.572786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:20.651866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebd7ea-1c6f-4e9d-89cc-c95e42582681 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:20.723503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:20.724150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:20.724430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:20.724704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:20.726361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:20.800168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:20.834079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:20.835541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eaad9f-226c-496c-8241-b0f7b1a94476 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:20.854361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 754/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:22.907686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83eb80-e91a-4587-a37e-c7201ca502fb None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:22.908228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83eb80-e91a-4587-a37e-c7201ca502fb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:22.908228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83eb80-e91a-4587-a37e-c7201ca502fb None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:22.909607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 731/1485] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:22.914751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec142cd0-0386-4b6e-bd43-0af9cdad5892 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:22.915235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec142cd0-0386-4b6e-bd43-0af9cdad5892 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:22.915373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec142cd0-0386-4b6e-bd43-0af9cdad5892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:22.978727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ec142cd0-0386-4b6e-bd43-0af9cdad5892 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:22.979919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ec142cd0-0386-4b6e-bd43-0af9cdad5892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:23.076658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 755/1486] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:23.113338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83eb80-e91a-4587-a37e-c7201ca502fb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:23.114911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4b83eb80-e91a-4587-a37e-c7201ca502fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:23.216968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83eb80-e91a-4587-a37e-c7201ca502fb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:23.293260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:23.295150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:23.295782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:23.296252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:23.296690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:23.298533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:23.404193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:23.405869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92558b49-d540-473c-9363-88be86a659e5 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:23.423726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 732/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:25.627691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba439-c0ea-44df-a0fc-242523b9bf21 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:25.628239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba439-c0ea-44df-a0fc-242523b9bf21 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:25.628239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aba439-c0ea-44df-a0fc-242523b9bf21 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:25.629646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 756/1488] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:25.635458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc05a0-ec9b-4202-aa0c-8c602a20d2d2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:25.635885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc05a0-ec9b-4202-aa0c-8c602a20d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:25.636000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc05a0-ec9b-4202-aa0c-8c602a20d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:25.699933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c7cc05a0-ec9b-4202-aa0c-8c602a20d2d2 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:25.701134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c7cc05a0-ec9b-4202-aa0c-8c602a20d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:25.751820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:25.795765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 733/1489] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:25.835937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba439-c0ea-44df-a0fc-242523b9bf21 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:25.838105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-66aba439-c0ea-44df-a0fc-242523b9bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:25.872751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:25.941380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aba439-c0ea-44df-a0fc-242523b9bf21 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:26.017255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:26.018166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:26.018551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:26.018953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:26.021097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:26.102442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:26.131662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:26.133103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fe25-3195-48d3-9f14-35e48df533a5 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:26.151372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 757/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:28.332484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc4cb6-37ce-437d-9419-493617366bdf None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:28.333026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc4cb6-37ce-437d-9419-493617366bdf None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:28.333123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc4cb6-37ce-437d-9419-493617366bdf None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:28.334466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 734/1491] 10.222.0.39 () {54 vars in 926 bytes} [Wed Jul 13 20:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 20:59:28.340237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c279-6ea8-4a21-94a4-d3edb918b8ec None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:28.340644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c279-6ea8-4a21-94a4-d3edb918b8ec None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:28.340800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c279-6ea8-4a21-94a4-d3edb918b8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:28.406288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9f23c279-6ea8-4a21-94a4-d3edb918b8ec None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 20:59:28.407465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9f23c279-6ea8-4a21-94a4-d3edb918b8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:28.501044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 758/1492] 10.222.0.39 () {58 vars in 1023 bytes} [Wed Jul 13 20:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 20:59:28.539775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc4cb6-37ce-437d-9419-493617366bdf None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 20:59:28.541853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9fcc4cb6-37ce-437d-9419-493617366bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:28.645033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc4cb6-37ce-437d-9419-493617366bdf None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 20:59:28.717552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 20:59:28.718130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 20:59:28.718414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 20:59:28.718714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 20:59:28.720345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 20:59:28.729384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:28.830276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 20:59:28.831744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aa031-3183-4e87-9453-48acdd095ace service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 20:59:28.849786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 735/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 20:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 20:59:38.499233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=82 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 20:59:38.847515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=50 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:53.722342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:53.722955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:53.722955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:53.795935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:00:53.797135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:53.891195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 759/1494] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:00:53.899307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:53.899804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:53.899910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:53.971985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:00:53.973222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:54.066240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 736/1495] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:00:54.071653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:54.072857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:54.084070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93285a3-40f1-4d9c-8e89-e6de991c7371 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:54.145946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:54.180625 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:54.181295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:54.181576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:54.181852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:54.182575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:00:54.182914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:54.183929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:54.183929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:00:54.183929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:00:54.184832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:54.195938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 760/1496] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:00:54.204105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:54.213347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd6834-4fab-4725-a770-20c2353b6381 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:54.274106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:54.310309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:54.311016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:54.311302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:54.311579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:54.312278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:54.313997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:54.313997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:00:54.313997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:00:54.315231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:54.367573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 737/1497] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:00:54 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:00:54.371759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:54.373300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:54.405140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9af3-770a-4dd8-aa5c-6f3a9a6efab3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:54.495480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:54.496135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:54.496415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:54.496687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:54.497357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:54.498359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:54.498359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:00:54.498359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:00:54.499604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:54.539026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 761/1498] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:00:54 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:00:54.542906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:54.544409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:54.605700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfa1a6-741e-4759-9bfb-3343a64fad87 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:54.688881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:54.701006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:54.701673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:54.701951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:54.702223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:54.702978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:54.703984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:54.703984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:00:54.703984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:00:54.704865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:54.717910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 738/1499] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:00:54.721923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:54.723511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:54.735440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12b2ac-ae17-4cb4-beb0-3e865bc36950 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:54.759638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:54.830562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:54.831258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:54.831542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:54.831817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:54.832544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:54.833557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:54.833557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:00:54.833557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:00:54.834443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe890ce8-ad92-49a0-8104-31042d8e5d3b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:54.847076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 762/1500] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:00:54.851218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:54.852740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:54.885430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f553dd3-644a-419e-b270-4ccf902a55b0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:54.977369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:54.978249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:54.978533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:54.978837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] PATH_INFO: `/v3/projects/1ac6054f16e94659b22f19193675b6aa/users/ae1d6d5ef77644ca976545ba1677315a/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.000646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac6054f16e94659b22f19193675b6aa, user_id=ae1d6d5ef77644ca976545ba1677315a, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:55.001673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:55.001673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:00:55.001673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:00:55.003661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22444a-a3b4-4642-af86-ef831e558868 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:55.029629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 739/1501] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:00:54 2022] PUT /identity/v3/projects/1ac6054f16e94659b22f19193675b6aa/users/ae1d6d5ef77644ca976545ba1677315a/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:00:55.041700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ddd23-f031-43eb-9539-4787e156f143 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:55.042237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ddd23-f031-43eb-9539-4787e156f143 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:55.042756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ddd23-f031-43eb-9539-4787e156f143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.097614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bb9ddd23-f031-43eb-9539-4787e156f143 None None] MFA Rules not processed for user `ae1d6d5ef77644ca976545ba1677315a`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:00:55.098771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bb9ddd23-f031-43eb-9539-4787e156f143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:55.174168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 763/1502] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:00:55.187751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad138a4-bfb9-4719-b5c8-18e2896efe3c None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:55.188039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad138a4-bfb9-4719-b5c8-18e2896efe3c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:55.188317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad138a4-bfb9-4719-b5c8-18e2896efe3c None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.189684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 740/1503] 10.222.0.39 () {54 vars in 963 bytes} [Wed Jul 13 21:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:00:55.195209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675186e-fbe1-475b-97f3-a113d766568c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:55.195490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675186e-fbe1-475b-97f3-a113d766568c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:55.195791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675186e-fbe1-475b-97f3-a113d766568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.225801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:55.271532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-7675186e-fbe1-475b-97f3-a113d766568c None None] MFA Rules not processed for user `9a58cb03e518413ab12cc84d793c4736`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:00:55.272756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7675186e-fbe1-475b-97f3-a113d766568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:55.347224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 764/1504] 10.222.0.39 () {58 vars in 1060 bytes} [Wed Jul 13 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:00:55.354894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad138a4-bfb9-4719-b5c8-18e2896efe3c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:55.356609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cad138a4-bfb9-4719-b5c8-18e2896efe3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:55.407788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:55.460897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad138a4-bfb9-4719-b5c8-18e2896efe3c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:55.532087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:55.532726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:55.533006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:55.533278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.534933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:55.631135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:55.632582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:55.635999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:55.650231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 741/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:00:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:00:55.692666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2464474-e09d-43b0-ab1f-1585acedf00c None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:55.692955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2464474-e09d-43b0-ab1f-1585acedf00c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:55.693258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2464474-e09d-43b0-ab1f-1585acedf00c None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.694588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 765/1506] 10.222.0.39 () {52 vars in 829 bytes} [Wed Jul 13 21:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:00:55.698051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:55.699326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:55.795722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950a321-849d-4b18-bcbe-fc7071810d0c service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:55.886467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:55.887290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:55.887568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:55.887840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] PATH_INFO: `/v3/users/ae1d6d5ef77644ca976545ba1677315a` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:55.899682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1d6d5ef77644ca976545ba1677315a)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:55.900697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:55.900697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:00:55.900697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:00:55.901899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:55.911790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:55.968257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1d6d5ef77644ca976545ba1677315a'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:55.978380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1d6d5ef77644ca976545ba1677315a'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:55.996624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1d6d5ef77644ca976545ba1677315a'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:55.997161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1d6d5ef77644ca976545ba1677315a'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:55.997607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1d6d5ef77644ca976545ba1677315a'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:56.014757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 742/1507] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:00:55 2022] DELETE /identity/v3/users/ae1d6d5ef77644ca976545ba1677315a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:00:56.019155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:56.019615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:56.019721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:56.076881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:00:56.078102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:56.152479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:56.166264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 766/1508] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:00:56.175722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:56.176934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:56.275006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e3053-cc4b-4d19-a6e9-62240173daa5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:56.330949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:00:56.351450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:56.352105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:56.352390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:56.352671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:56.354320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:56.459714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:56.461708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc8ef7-440d-4753-9406-23fd616425bd service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:56.479716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 743/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:00:56.832743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:00:56.834089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:00:56.845675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a220-3a0b-420c-beb0-3754e9acea8e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:00:56.943069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:00:56.943923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:00:56.944204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:00:56.945102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] PATH_INFO: `/v3/projects/1ac6054f16e94659b22f19193675b6aa` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:00:56.954823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac6054f16e94659b22f19193675b6aa)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:00:56.955842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:00:56.955842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:00:56.955842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:00:56.957042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:00:57.012670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac6054f16e94659b22f19193675b6aa', 'request_id': 'req-cf239452-e451-4f3c-b428-4ca52897e3d2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:57.026544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac6054f16e94659b22f19193675b6aa', 'request_id': 'req-cf239452-e451-4f3c-b428-4ca52897e3d2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:57.032683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac6054f16e94659b22f19193675b6aa', 'request_id': 'req-cf239452-e451-4f3c-b428-4ca52897e3d2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:57.058924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] The token cache is being invalidate because project 1ac6054f16e94659b22f19193675b6aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:00:57.059209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:57.059718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cf239452-e451-4f3c-b428-4ca52897e3d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:00:57.061065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 767/1510] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:00:56 2022] DELETE /identity/v3/projects/1ac6054f16e94659b22f19193675b6aa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:01:06.477477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=62 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:01:07.056954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:02:31.451600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6fed5-ab95-4315-bf4f-919abe3999c4 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:31.452196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6fed5-ab95-4315-bf4f-919abe3999c4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:31.452196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6fed5-ab95-4315-bf4f-919abe3999c4 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:31.453545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 21:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:31.458266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23056827-08e4-460c-b067-ef1479c2e178 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:31.458692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23056827-08e4-460c-b067-ef1479c2e178 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:31.458850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23056827-08e4-460c-b067-ef1479c2e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:31.537872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-23056827-08e4-460c-b067-ef1479c2e178 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:31.539144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-23056827-08e4-460c-b067-ef1479c2e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:31.641040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:31.793290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6fed5-ab95-4315-bf4f-919abe3999c4 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:31.795430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2da6fed5-ab95-4315-bf4f-919abe3999c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:31.911686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6fed5-ab95-4315-bf4f-919abe3999c4 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:31.917771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55abdbe-b06a-46f0-afb2-f94cae0446aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:31.918418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55abdbe-b06a-46f0-afb2-f94cae0446aa service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:31.918746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55abdbe-b06a-46f0-afb2-f94cae0446aa service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:31.919029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55abdbe-b06a-46f0-afb2-f94cae0446aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:31.927022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55abdbe-b06a-46f0-afb2-f94cae0446aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:31.928601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55abdbe-b06a-46f0-afb2-f94cae0446aa service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:31.937836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 745/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:32.762990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dd0e6-c2cc-4147-8d5f-a2910c716a57 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:32.763534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dd0e6-c2cc-4147-8d5f-a2910c716a57 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:32.763534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dd0e6-c2cc-4147-8d5f-a2910c716a57 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:32.764973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 21:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:32.769633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f284-a18e-479c-ae95-3466d22b9fd6 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:32.770047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f284-a18e-479c-ae95-3466d22b9fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:32.770191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9f284-a18e-479c-ae95-3466d22b9fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:32.789274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4ef9f284-a18e-479c-ae95-3466d22b9fd6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:32.790456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4ef9f284-a18e-479c-ae95-3466d22b9fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:32.798486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:32.947613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dd0e6-c2cc-4147-8d5f-a2910c716a57 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:32.962272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dd0e6-c2cc-4147-8d5f-a2910c716a57 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:32.967702 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601163ce-8fb5-452f-ae33-719eddb70388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:32.968339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601163ce-8fb5-452f-ae33-719eddb70388 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:32.968622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601163ce-8fb5-452f-ae33-719eddb70388 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:32.968903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601163ce-8fb5-452f-ae33-719eddb70388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:32.977744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601163ce-8fb5-452f-ae33-719eddb70388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:32.979394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601163ce-8fb5-452f-ae33-719eddb70388 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:32.988721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 770/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:33.741159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd058a-ebb3-48f2-80f0-30cf17cfba88 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:33.741445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd058a-ebb3-48f2-80f0-30cf17cfba88 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:33.741721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd058a-ebb3-48f2-80f0-30cf17cfba88 None None] PATH_INFO: `/v3` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:33.743078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 747/1517] 10.222.0.26 () {54 vars in 925 bytes} [Wed Jul 13 21:02:33 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:02:33.749228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa672d-57d3-4a84-a26a-eca810e1ac3c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:33.749525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa672d-57d3-4a84-a26a-eca810e1ac3c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:33.749810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa672d-57d3-4a84-a26a-eca810e1ac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:33.807732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-d3aa672d-57d3-4a84-a26a-eca810e1ac3c None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:33.808934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d3aa672d-57d3-4a84-a26a-eca810e1ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:33.825285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:02:33.879946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 771/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Jul 13 21:02:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:42.784005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=56 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:02:42.824861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8d672-720d-4d6f-9355-3d89cc8cfc8c None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:42.825075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8d672-720d-4d6f-9355-3d89cc8cfc8c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:42.825354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8d672-720d-4d6f-9355-3d89cc8cfc8c None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:42.826757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 748/1519] 10.222.0.39 () {54 vars in 960 bytes} [Wed Jul 13 21:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:42.831852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c24c9-1606-4c5d-8829-06b465007ad8 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:42.832308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c24c9-1606-4c5d-8829-06b465007ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:42.832453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c24c9-1606-4c5d-8829-06b465007ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:42.851709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e61c24c9-1606-4c5d-8829-06b465007ad8 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:42.853529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e61c24c9-1606-4c5d-8829-06b465007ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:42.862071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 772/1520] 10.222.0.39 () {58 vars in 1057 bytes} [Wed Jul 13 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:42.869492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8d672-720d-4d6f-9355-3d89cc8cfc8c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:42.882568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8d672-720d-4d6f-9355-3d89cc8cfc8c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:42.887467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd2c8f-0358-40a2-8ea6-72da70384707 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:42.888090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd2c8f-0358-40a2-8ea6-72da70384707 service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:42.888371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd2c8f-0358-40a2-8ea6-72da70384707 service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:42.888647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd2c8f-0358-40a2-8ea6-72da70384707 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:42.896726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd2c8f-0358-40a2-8ea6-72da70384707 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:42.898289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd2c8f-0358-40a2-8ea6-72da70384707 service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:42.908455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 749/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:44.225318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed811427-c585-4b3c-ba29-07f4b1244276 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:44.225881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed811427-c585-4b3c-ba29-07f4b1244276 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:44.225881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed811427-c585-4b3c-ba29-07f4b1244276 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:44.227828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 21:02:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:44.232817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc705fc-5035-490a-9d2f-5c1ca581fbff None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:44.233247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc705fc-5035-490a-9d2f-5c1ca581fbff None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:44.233354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc705fc-5035-490a-9d2f-5c1ca581fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:44.251723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6fc705fc-5035-490a-9d2f-5c1ca581fbff None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:44.252915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6fc705fc-5035-490a-9d2f-5c1ca581fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:44.260800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 21:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:44.416462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed811427-c585-4b3c-ba29-07f4b1244276 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:44.430536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed811427-c585-4b3c-ba29-07f4b1244276 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:44.436257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfae333-d1c0-41f3-99fa-9c79d291af33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:44.436903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae333-d1c0-41f3-99fa-9c79d291af33 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:44.437189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae333-d1c0-41f3-99fa-9c79d291af33 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:44.437476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfae333-d1c0-41f3-99fa-9c79d291af33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:44.445657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae333-d1c0-41f3-99fa-9c79d291af33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:44.447279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfae333-d1c0-41f3-99fa-9c79d291af33 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:44.456659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 774/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:45.270035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc5650-1b6f-4871-86d7-2d9437807985 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:45.270576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc5650-1b6f-4871-86d7-2d9437807985 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:45.270576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc5650-1b6f-4871-86d7-2d9437807985 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:45.272160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 21:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:45.276967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b627e-27fd-488e-ba4e-7f4ffe9a7ebb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:45.277350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b627e-27fd-488e-ba4e-7f4ffe9a7ebb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:45.277523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b627e-27fd-488e-ba4e-7f4ffe9a7ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:45.295451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-030b627e-27fd-488e-ba4e-7f4ffe9a7ebb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:45.296618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-030b627e-27fd-488e-ba4e-7f4ffe9a7ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:45.304339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:45.454870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc5650-1b6f-4871-86d7-2d9437807985 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:45.468969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc5650-1b6f-4871-86d7-2d9437807985 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:45.474428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8997b8c-8424-41d5-8a2b-67a128a221e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:45.475094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8997b8c-8424-41d5-8a2b-67a128a221e0 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:45.475384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8997b8c-8424-41d5-8a2b-67a128a221e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:45.475661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8997b8c-8424-41d5-8a2b-67a128a221e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:45.484169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8997b8c-8424-41d5-8a2b-67a128a221e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:45.485741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8997b8c-8424-41d5-8a2b-67a128a221e0 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:45.495750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 752/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:49.351835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f2c93-503b-499e-a001-73c3c224926f None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:49.352363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f2c93-503b-499e-a001-73c3c224926f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:49.352363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f2c93-503b-499e-a001-73c3c224926f None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:49.353912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 21:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:49.359182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc50a1-c9cd-4de5-88ac-df575a59d4fa None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:49.359605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc50a1-c9cd-4de5-88ac-df575a59d4fa None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:49.359770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc50a1-c9cd-4de5-88ac-df575a59d4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:49.377523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-50fc50a1-c9cd-4de5-88ac-df575a59d4fa None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:49.378763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-50fc50a1-c9cd-4de5-88ac-df575a59d4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:49.387098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:49.459026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f2c93-503b-499e-a001-73c3c224926f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:49.460751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a50f2c93-503b-499e-a001-73c3c224926f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:49.537612 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f2c93-503b-499e-a001-73c3c224926f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:49.541621 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbbb4c-0b0a-4a1f-b3d8-4b567a9ad79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:49.542259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbbb4c-0b0a-4a1f-b3d8-4b567a9ad79c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:49.542547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbbb4c-0b0a-4a1f-b3d8-4b567a9ad79c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:49.542848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbbb4c-0b0a-4a1f-b3d8-4b567a9ad79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:49.551597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbbb4c-0b0a-4a1f-b3d8-4b567a9ad79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:49.553004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbbb4c-0b0a-4a1f-b3d8-4b567a9ad79c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:49.562128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 777/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:51.553348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d0c2e-49bd-459e-97c6-050183955552 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:51.553872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d0c2e-49bd-459e-97c6-050183955552 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:51.553872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d0c2e-49bd-459e-97c6-050183955552 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:51.555230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 21:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:51.560292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec3b3f-6020-4055-abbc-25c9e6738715 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:51.560694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec3b3f-6020-4055-abbc-25c9e6738715 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:51.560907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec3b3f-6020-4055-abbc-25c9e6738715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:51.579388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-42ec3b3f-6020-4055-abbc-25c9e6738715 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:51.580580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-42ec3b3f-6020-4055-abbc-25c9e6738715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:51.589079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:51.623949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d0c2e-49bd-459e-97c6-050183955552 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:51.635053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d0c2e-49bd-459e-97c6-050183955552 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:51.638512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116891dd-f5d1-423d-a1c2-0add610ce761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:51.639177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116891dd-f5d1-423d-a1c2-0add610ce761 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:51.639461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116891dd-f5d1-423d-a1c2-0add610ce761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:51.639734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116891dd-f5d1-423d-a1c2-0add610ce761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:51.648417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116891dd-f5d1-423d-a1c2-0add610ce761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:51.649850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116891dd-f5d1-423d-a1c2-0add610ce761 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:51.660408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 755/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:55.213783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b70ef-ca4f-4903-8ba9-e6ac5abdca10 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:55.214311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b70ef-ca4f-4903-8ba9-e6ac5abdca10 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:55.214311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b70ef-ca4f-4903-8ba9-e6ac5abdca10 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:55.215670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 21:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:55.220908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d76b0-0eac-46ca-990e-2b367a1ebae9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:55.221366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d76b0-0eac-46ca-990e-2b367a1ebae9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:55.221461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d76b0-0eac-46ca-990e-2b367a1ebae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:55.240466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d94d76b0-0eac-46ca-990e-2b367a1ebae9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:55.242004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d94d76b0-0eac-46ca-990e-2b367a1ebae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:55.250637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:55.560298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b70ef-ca4f-4903-8ba9-e6ac5abdca10 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:55.561980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-189b70ef-ca4f-4903-8ba9-e6ac5abdca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:55.625706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:02:55.639242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b70ef-ca4f-4903-8ba9-e6ac5abdca10 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:55.642750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663ad92-f6c3-4036-a5d1-9281577674d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:55.643397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ad92-f6c3-4036-a5d1-9281577674d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:55.643682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ad92-f6c3-4036-a5d1-9281577674d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:55.643960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ad92-f6c3-4036-a5d1-9281577674d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:55.653569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663ad92-f6c3-4036-a5d1-9281577674d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:55.655021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663ad92-f6c3-4036-a5d1-9281577674d9 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:55.664225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 780/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:02:57.694939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607b294-ca6d-4546-b956-a1259f94d727 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:57.695502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607b294-ca6d-4546-b956-a1259f94d727 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:57.695502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607b294-ca6d-4546-b956-a1259f94d727 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:57.696868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 21:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:02:57.702090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acfd6d-22af-4295-803a-c9093b3148bb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:57.702517 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acfd6d-22af-4295-803a-c9093b3148bb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:57.702680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acfd6d-22af-4295-803a-c9093b3148bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:57.721015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-98acfd6d-22af-4295-803a-c9093b3148bb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:02:57.722208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-98acfd6d-22af-4295-803a-c9093b3148bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:57.730857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:02:58.028730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607b294-ca6d-4546-b956-a1259f94d727 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:02:58.030432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5607b294-ca6d-4546-b956-a1259f94d727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:02:58.043605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607b294-ca6d-4546-b956-a1259f94d727 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:02:58.047083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd088bc0-d131-4b49-906c-8caa991f1a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:02:58.047715 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd088bc0-d131-4b49-906c-8caa991f1a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:02:58.047995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd088bc0-d131-4b49-906c-8caa991f1a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:02:58.048271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd088bc0-d131-4b49-906c-8caa991f1a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:02:58.057497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd088bc0-d131-4b49-906c-8caa991f1a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:02:58.058937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd088bc0-d131-4b49-906c-8caa991f1a82 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:02:58.068102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 758/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:03:07.716296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=11 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:03:08.064006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=33 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:26.940548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2c678-7caf-418c-9bc6-7247392e41ac None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:26.941179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2c678-7caf-418c-9bc6-7247392e41ac None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:26.941179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2c678-7caf-418c-9bc6-7247392e41ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:26.963996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0fc2c678-7caf-418c-9bc6-7247392e41ac None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:26.965643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0fc2c678-7caf-418c-9bc6-7247392e41ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:26.979108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 782/1540] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:04:26.986528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec799-17cf-45e9-a796-ffe9369fe113 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:26.987563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec799-17cf-45e9-a796-ffe9369fe113 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:26.988042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ec799-17cf-45e9-a796-ffe9369fe113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:26.991566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:26.992081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:26.992540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.018610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.020151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.020786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-424ec799-17cf-45e9-a796-ffe9369fe113 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.022212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-424ec799-17cf-45e9-a796-ffe9369fe113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.036133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 783/1541] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.041334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.044872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 759/1542] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.051959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9ce81-fc5f-45f9-bfcf-b64ddb47fff0 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.061541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.062601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.063133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.063614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.064582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.065101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.066981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.069831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.070318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.071791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.083304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 784/1543] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:04:27.094639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.104832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.106584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.117518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcce42-30f8-42d2-9868-0aad00f4862e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.127619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 760/1544] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 13 21:04:27.132409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.132712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.133288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.133684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.134059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.134878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.135998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:04:27.135998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:04:27.135998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:04:27.137440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.144938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7010bf-052d-4863-90f6-204a732156f8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.166449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.167811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.168282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.168737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.169703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.170176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.172797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:04:27.172797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:04:27.172797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:04:27.174031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.199259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 761/1545] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:04:27.204475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.220791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95e562-c33d-4777-ad7d-1737316dd7d9 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.223023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 785/1546] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:04:27.227651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.331935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d2be1-dc4d-4730-b5fd-52fa5f67fc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.332617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d2be1-dc4d-4730-b5fd-52fa5f67fc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.333041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d2be1-dc4d-4730-b5fd-52fa5f67fc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.333312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d2be1-dc4d-4730-b5fd-52fa5f67fc6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.333936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d2be1-dc4d-4730-b5fd-52fa5f67fc6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.335909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d2be1-dc4d-4730-b5fd-52fa5f67fc6b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.350002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd5658-da1b-4e70-a27e-07f51813e47c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.357306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46600fe5-74b2-4b39-9e6e-592465afcead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.358919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600fe5-74b2-4b39-9e6e-592465afcead admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.359654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600fe5-74b2-4b39-9e6e-592465afcead admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.360191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46600fe5-74b2-4b39-9e6e-592465afcead admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.361077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46600fe5-74b2-4b39-9e6e-592465afcead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.363331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46600fe5-74b2-4b39-9e6e-592465afcead admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.367367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 762/1547] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:04:27.368867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf3e9e-2650-45a3-806c-ca2401ed09ae None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.369162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf3e9e-2650-45a3-806c-ca2401ed09ae None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.369444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf3e9e-2650-45a3-806c-ca2401ed09ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.386266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 786/1548] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:04:27.386266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170df0-3509-400b-bf1c-5c5cf48d521f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.386266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170df0-3509-400b-bf1c-5c5cf48d521f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.386266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb170df0-3509-400b-bf1c-5c5cf48d521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.390708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-00cf3e9e-2650-45a3-806c-ca2401ed09ae None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.391950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-00cf3e9e-2650-45a3-806c-ca2401ed09ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.407279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cb170df0-3509-400b-bf1c-5c5cf48d521f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.408453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cb170df0-3509-400b-bf1c-5c5cf48d521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.450152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 763/1549] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.451712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bb8e9-2667-46ba-9771-4bcf53a6eaf5 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.452004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bb8e9-2667-46ba-9771-4bcf53a6eaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.452283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bb8e9-2667-46ba-9771-4bcf53a6eaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.470531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fc5bb8e9-2667-46ba-9771-4bcf53a6eaf5 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.471836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fc5bb8e9-2667-46ba-9771-4bcf53a6eaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.473600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 787/1550] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.474902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565ec35-e08e-4b7c-828f-e6ad38593011 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.475790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565ec35-e08e-4b7c-828f-e6ad38593011 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.476071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565ec35-e08e-4b7c-828f-e6ad38593011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.481044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 764/1551] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.487993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.488307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.488591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.494545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e565ec35-e08e-4b7c-828f-e6ad38593011 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.495795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e565ec35-e08e-4b7c-828f-e6ad38593011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.503790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 788/1552] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.504990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.505286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.505549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.506180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.507564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.516031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 765/1553] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.516786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.523687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.524970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.528804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b3819-289a-4214-af57-4ea46dc77c27 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.533283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 789/1554] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.533530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.534083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.534190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.534426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.534676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.535365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.536539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:04:27.536539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:04:27.536539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:04:27.538359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6ce09-c374-49a1-988a-2b0da4491dd5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.543914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc18517-b843-4d83-a236-d217684a0740 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.548843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2167d-41d2-49c8-a6b9-0380b5788f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.549555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2167d-41d2-49c8-a6b9-0380b5788f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.549849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2167d-41d2-49c8-a6b9-0380b5788f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.550181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2167d-41d2-49c8-a6b9-0380b5788f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.551048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2167d-41d2-49c8-a6b9-0380b5788f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.552866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2167d-41d2-49c8-a6b9-0380b5788f9b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.560404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 766/1555] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:27.561882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189492a9-5577-459f-b9ed-3ad17ac8a3d1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.562172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189492a9-5577-459f-b9ed-3ad17ac8a3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.562449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189492a9-5577-459f-b9ed-3ad17ac8a3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.567130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 790/1556] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:27.567130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e0cc-92cc-472c-b3f4-ab8ffe4ec63c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.567130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e0cc-92cc-472c-b3f4-ab8ffe4ec63c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.567130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1e0cc-92cc-472c-b3f4-ab8ffe4ec63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.582965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-189492a9-5577-459f-b9ed-3ad17ac8a3d1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.584218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-189492a9-5577-459f-b9ed-3ad17ac8a3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.585610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-abd1e0cc-92cc-472c-b3f4-ab8ffe4ec63c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.586804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-abd1e0cc-92cc-472c-b3f4-ab8ffe4ec63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.592877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 767/1557] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.594137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941bb5a-c838-4ab9-9646-2b2fab795d34 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.594420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941bb5a-c838-4ab9-9646-2b2fab795d34 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.594724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941bb5a-c838-4ab9-9646-2b2fab795d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.598856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 791/1558] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.600183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d1a0f-7c26-409a-8f21-9ee129d85af7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.601166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d1a0f-7c26-409a-8f21-9ee129d85af7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.601474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d1a0f-7c26-409a-8f21-9ee129d85af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.607685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:27.615458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b941bb5a-c838-4ab9-9646-2b2fab795d34 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.616953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b941bb5a-c838-4ab9-9646-2b2fab795d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.620906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f86d1a0f-7c26-409a-8f21-9ee129d85af7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.622071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f86d1a0f-7c26-409a-8f21-9ee129d85af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.624769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 768/1559] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.625939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.626323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.626604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.633033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 792/1560] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.634600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.634934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.635214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.648876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.650051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.654469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:27.655680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:27.661867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 769/1561] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.661867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.666940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 793/1562] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:27.667773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.668646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7a47a-3914-4bb6-b845-bb64975c649a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.673422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.674058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.674339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.674630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.675362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.676946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.677062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0418536-88b4-4af0-ab51-8557d31a6bbc None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.679210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:27.681522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.682213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.682496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.682806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.683896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.685092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.691477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 770/1563] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:27.691477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.696830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 794/1564] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:27.697683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.699408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbce529-2b4b-4317-920a-9174c2067f0f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.703964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.704616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.704889 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.705165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.705879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.706184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.707175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d00e8-73ae-47a6-9ead-7db7de316b74 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.707786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.712178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.712801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.713081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.713357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.714095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.714396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.715991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.720061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 771/1565] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:27.720898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.735475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 795/1566] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:27.735475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.738824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3dc8b-aeef-478c-b577-3733c04525ea admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.740594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9e7d-298b-4365-b88b-dcd9d6c73fac admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.743449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.744117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.744401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.744681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.745269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.745468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.745725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.745921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.746231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.746501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.747290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.747409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.747594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.749194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.759478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 796/1567] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:27.760090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.768660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 772/1568] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:27.769341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9200-ab6e-44b5-8313-bbb5e0092228 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.769456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.774836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.775487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.776055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.776335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.777057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.777396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.779077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.782099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93851412-dcc2-410a-9f0d-2c1e9ce78868 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.787273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.787897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.788632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.788917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.789867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:27.790176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.791179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 797/1569] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:27.791793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.791956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.802485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f65e6a-7a62-4a30-96e1-00bdf01b207f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.804823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 773/1570] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:27.805609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.807580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.808228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.808513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.808788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.809589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.811196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.816559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af1444-6312-4ee1-b024-447c0a191ca7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.821539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.824991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 798/1571] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:27.825132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.825217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.825293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.825293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] PATH_INFO: `/v3/projects/0a2703a8b75043e0ab0656482fa9ddeb/users/d9cab6f67cd34c46886d036ba170331e/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.831925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214bf4b-6aa5-4781-8217-c4e816b7dd5c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.837183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.837820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.838109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.838390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.839118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.840723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2703a8b75043e0ab0656482fa9ddeb, user_id=d9cab6f67cd34c46886d036ba170331e, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.841016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.843403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.856797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 774/1572] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:27 2022] PUT /identity/v3/projects/0a2703a8b75043e0ab0656482fa9ddeb/users/d9cab6f67cd34c46886d036ba170331e/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:27.857720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.869381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 799/1573] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:27.870266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.919845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8fc22-e3e4-40d5-a11b-f8dbb087b227 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.931164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee328ce5-71ad-4a12-991a-8e4a679c427d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:27.935807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.936481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.936763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.937038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.937718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.939654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:27.969408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 800/1574] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:27.969408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:27.979914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:27.980580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:27.980866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:27.981146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:27.981846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:27.983781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.010168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 775/1575] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.011900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.023949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:28.037434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d3713-eb2c-4d57-8e39-a573c6beec30 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.041613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:28.073872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71862cc-b217-4ed5-8e71-60d231ac944d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.079175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.079898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.080186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.080468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.081167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.083154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.093421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.094079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.094361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.094661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.095352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.097249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.109521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 776/1576] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.110581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.129490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 801/1577] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:27 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.129490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.175850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08d0e7-b001-47c6-b594-425a39aa55cf admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.198145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96acb7-7df5-4828-8e12-dcd148bb4f26 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.202778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.203681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.204027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.204310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] PATH_INFO: `/v3/projects/9eb5da85f8a74b08aef7b37cc209ca92/users/a28051d0f956461c8f3b39825e33af96/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.219081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb5da85f8a74b08aef7b37cc209ca92, user_id=a28051d0f956461c8f3b39825e33af96, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.220123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:04:28.220123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:04:28.220123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:04:28.222066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034cadd-0db3-4920-9895-c5eca9a1b20d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.232486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.233173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.233456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.233730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.234414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.235328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 802/1578] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:28 2022] PUT /identity/v3/projects/9eb5da85f8a74b08aef7b37cc209ca92/users/a28051d0f956461c8f3b39825e33af96/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:28.236435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.236620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372742be-a031-4221-8661-631d58992a2c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.236881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372742be-a031-4221-8661-631d58992a2c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.237160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372742be-a031-4221-8661-631d58992a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.254181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-372742be-a031-4221-8661-631d58992a2c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:28.255657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-372742be-a031-4221-8661-631d58992a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:28.266560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 777/1579] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.266560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.311381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:28.321144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 803/1580] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:28.321854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372742be-a031-4221-8661-631d58992a2c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.329631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe21d1-805d-4a28-a269-8d28a48c2c59 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.331392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372742be-a031-4221-8661-631d58992a2c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.335027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.335857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.335969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.336169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.336447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.336659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.336902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.337144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.337256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.337840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.339095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.339778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.346310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:28.362403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 778/1581] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.362403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.362729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 804/1582] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/users => generated 283 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.362729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.370136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b393d-ec00-4a5b-9bf8-2f1c02448daa admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.373029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3c70b-e3ec-48b0-baa8-c748edbe5ba9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.374758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.375419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.375709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.375988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.376674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.377893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ee932-ee0b-4da7-8880-f1e3bf336087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.378537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ee932-ee0b-4da7-8880-f1e3bf336087 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.378667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.378851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ee932-ee0b-4da7-8880-f1e3bf336087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.379128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ee932-ee0b-4da7-8880-f1e3bf336087 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.379805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ee932-ee0b-4da7-8880-f1e3bf336087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.382342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ee932-ee0b-4da7-8880-f1e3bf336087 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.403206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 779/1583] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/users => generated 287 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.403206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.403458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 805/1584] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.404456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.404746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.405027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.410510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703f082-7286-4257-9690-a2d4d3467e8d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.415339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.416029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.416336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.416607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.417319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.419735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.423370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:28.424565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:28.432828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 806/1585] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:28.433576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.441496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 780/1586] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/users => generated 291 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.442315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.442784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956668e4-dc8b-4fe2-a383-5d425d15dacd None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.448187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.448848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.449129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.449401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.450085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.452032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.472062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 807/1587] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:28 2022] POST /identity/v3/users => generated 292 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:28.472860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.482644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3d14c-0182-40fb-9d51-0e7eaac93e39 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.487351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.487988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.488268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.488539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.489251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.491386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.501002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 808/1588] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.504541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78b2a7-b990-4da9-8da4-29d03d69e9ab admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.504790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.508204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.508854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.509193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.509529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.514153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e00b7-ba4e-4443-a794-461ba8b2e2c0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.518748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.519022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.519364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.519641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.519908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.520454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.520622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.522156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.530193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 781/1589] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:28.531056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.542826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910266c-84ed-48b1-8254-058ad778da3d service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.548265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.549269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.549722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.550160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.551120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.552985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.569220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 782/1590] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.569220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.580389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5aaee5-face-4b03-b5db-51d95a10a6f2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.585530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.586450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.586784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.587067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.588061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.591706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.611500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 783/1591] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.611500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.613484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:28.631542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c9712-0eba-4993-b0ff-9c07b738e8b1 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.637563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.639662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.639961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.640296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.654324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.655575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 809/1592] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.656964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.659493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.669465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 784/1593] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:28.670300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.671504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814783d-1908-42f3-9d8e-115b09d25fbd admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.676742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.677541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.677853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.678218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.679924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.681552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.682175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d64c84-ff69-4a84-a413-786614d0040b service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.687370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.688126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.688503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.688899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.689712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.691559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.696211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 810/1594] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.697420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.704939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 785/1595] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.705822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.720266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec1ac3-0fce-41cd-a058-cd7cf08a88aa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.723187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2affdf6-684b-484c-92e3-b0c0ca646248 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.728335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.729308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.730226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.730523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.730933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.731220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.731501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.731704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.732236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.732968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.734255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.734979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.751853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 786/1596] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.752727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.753461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 811/1597] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.754398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.767804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace9df5-529b-4c87-acb9-fa2d6e03c3a4 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.770665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fc8f4-4d99-4299-96e2-ce7e9d7a7948 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.774372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.775401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.776700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.777047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.777825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.778445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.779268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.779562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.779907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.780475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.781751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.783521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.794644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 812/1598] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.796805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 787/1599] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.798574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.798757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.813312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ced7d-60c4-4e58-9db2-8ba8e16e2c71 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.815678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9dc8e-8d11-4c80-859f-995a04594131 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.821384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.822571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.822996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.823425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.824561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.824971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.825505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.825851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.826199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.827093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.829703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.829795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.847124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 813/1600] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.848454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.860436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 788/1601] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:28.861518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.871707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12318713-bb2c-4fc4-8705-13e54c93ed23 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.874349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932009-960a-4b7e-9150-aa168e7651e4 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:28.877416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.878397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.878727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.879200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] PATH_INFO: `/v3/projects/93560bcf2e774cfa90a37675d206c510/users/a5f7d2d0c8cc4a8db378ca436b0b56b2/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.880960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:28.882404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:28.883908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:28.884233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] PATH_INFO: `/v3/projects/91bc9f9f4bc04202aabc75fc4bb9a6b2/users/fc4afeb23348438eabfe22f1a975f9ab/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:28.900187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93560bcf2e774cfa90a37675d206c510, user_id=a5f7d2d0c8cc4a8db378ca436b0b56b2, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.903376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.904567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91bc9f9f4bc04202aabc75fc4bb9a6b2, user_id=fc4afeb23348438eabfe22f1a975f9ab, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:28.908482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:28.922867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:28.928441 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 814/1602] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:28 2022] PUT /identity/v3/projects/93560bcf2e774cfa90a37675d206c510/users/a5f7d2d0c8cc4a8db378ca436b0b56b2/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:28.929455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:28.939365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 789/1603] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:28 2022] PUT /identity/v3/projects/91bc9f9f4bc04202aabc75fc4bb9a6b2/users/fc4afeb23348438eabfe22f1a975f9ab/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:28.944126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.014301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e0476-2069-459e-b9ec-2d92c1fb768b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.030409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001fda4-ad85-43c6-85c6-5c19ccaa602e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.035212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.036097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.036379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.036626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] PATH_INFO: `/v3/projects/e43ca8a5c9f2494fbb9c5c5509c5c79b/users/0884a33a47e448e7826f257880a28964/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.042858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:INSERT=10,keystone:SELECT=90 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:29.056593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43ca8a5c9f2494fbb9c5c5509c5c79b, user_id=0884a33a47e448e7826f257880a28964, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.059735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.075674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 790/1604] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:28 2022] PUT /identity/v3/projects/e43ca8a5c9f2494fbb9c5c5509c5c79b/users/0884a33a47e448e7826f257880a28964/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:29.076640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.106184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.107514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.108001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.108472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] PATH_INFO: `/v3/projects/ed45078466534951ba448cfaa895ad1d/users/cc27492e934840bdbb6dd179306988d6/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.128610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed45078466534951ba448cfaa895ad1d, user_id=cc27492e934840bdbb6dd179306988d6, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.132085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.142846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969e546-baf7-4cc7-9497-0de62f93c883 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.157033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 815/1605] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:28 2022] PUT /identity/v3/projects/ed45078466534951ba448cfaa895ad1d/users/cc27492e934840bdbb6dd179306988d6/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:29.157033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.222062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a17f8-6224-4912-9f94-34d461b9098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.223070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a17f8-6224-4912-9f94-34d461b9098a admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.223363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a17f8-6224-4912-9f94-34d461b9098a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.223628 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a17f8-6224-4912-9f94-34d461b9098a admin admin] PATH_INFO: `/v3/projects/c317c7bdea394bcdbc0657fadedac2f7/users/4711f2212b2b42b1bfe414d2795c6356/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.239012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d41468-2661-4f37-8da9-c1a4fb5bfc2c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.240158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a17f8-6224-4912-9f94-34d461b9098a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c317c7bdea394bcdbc0657fadedac2f7, user_id=4711f2212b2b42b1bfe414d2795c6356, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.243060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67a17f8-6224-4912-9f94-34d461b9098a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.244377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd3f12-4be0-4437-9f50-734693850eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.245414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd3f12-4be0-4437-9f50-734693850eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.245697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd3f12-4be0-4437-9f50-734693850eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.246648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd3f12-4be0-4437-9f50-734693850eb2 admin admin] PATH_INFO: `/v3/projects/95155d588e674097b7b73aad68e3c651/users/b4b83f5475f04e6d8435eae9da442d43/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.273099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 791/1606] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:29 2022] PUT /identity/v3/projects/c317c7bdea394bcdbc0657fadedac2f7/users/4711f2212b2b42b1bfe414d2795c6356/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:29.273099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45835d4e-c6a5-4b16-b53c-43ba084c33fe None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.273099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45835d4e-c6a5-4b16-b53c-43ba084c33fe None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.273099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45835d4e-c6a5-4b16-b53c-43ba084c33fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.276125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd3f12-4be0-4437-9f50-734693850eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95155d588e674097b7b73aad68e3c651, user_id=b4b83f5475f04e6d8435eae9da442d43, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.278917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd3f12-4be0-4437-9f50-734693850eb2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.293766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:29.294337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 816/1607] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:29 2022] PUT /identity/v3/projects/95155d588e674097b7b73aad68e3c651/users/b4b83f5475f04e6d8435eae9da442d43/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:29.304588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76100caa-8599-4749-a61d-dbfefd3d9196 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.304588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76100caa-8599-4749-a61d-dbfefd3d9196 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.304588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76100caa-8599-4749-a61d-dbfefd3d9196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.316135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-45835d4e-c6a5-4b16-b53c-43ba084c33fe None None] MFA Rules not processed for user `a5f7d2d0c8cc4a8db378ca436b0b56b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.317619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-45835d4e-c6a5-4b16-b53c-43ba084c33fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.326287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-76100caa-8599-4749-a61d-dbfefd3d9196 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.327812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-76100caa-8599-4749-a61d-dbfefd3d9196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.395276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 817/1608] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:29.396825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d6e4-75f2-4cf5-b4c3-2f655692c791 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.397673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d6e4-75f2-4cf5-b4c3-2f655692c791 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.397958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d6e4-75f2-4cf5-b4c3-2f655692c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.400234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 792/1609] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:29.401967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.402392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.402934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.421972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5bc5d6e4-75f2-4cf5-b4c3-2f655692c791 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.422567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5bc5d6e4-75f2-4cf5-b4c3-2f655692c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.436084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 818/1610] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:29.436084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a0eb-f780-4bd8-bf22-979dedfe6f14 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.436084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a0eb-f780-4bd8-bf22-979dedfe6f14 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.436453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8a0eb-f780-4bd8-bf22-979dedfe6f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.471583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b2e8a0eb-f780-4bd8-bf22-979dedfe6f14 None None] MFA Rules not processed for user `4711f2212b2b42b1bfe414d2795c6356`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.472866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b2e8a0eb-f780-4bd8-bf22-979dedfe6f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.485845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] MFA Rules not processed for user `cc27492e934840bdbb6dd179306988d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.488776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.491737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:29.565997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 819/1611] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:29.567952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1521bb9-aeb2-4116-b2a3-4947d0a8a340 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.568503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1521bb9-aeb2-4116-b2a3-4947d0a8a340 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.568605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1521bb9-aeb2-4116-b2a3-4947d0a8a340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.607421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 793/1612] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:29.609465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f1521bb9-aeb2-4116-b2a3-4947d0a8a340 None None] MFA Rules not processed for user `b4b83f5475f04e6d8435eae9da442d43`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.611377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f1521bb9-aeb2-4116-b2a3-4947d0a8a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.618130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.709228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93125abf-a548-4465-b2ef-310a23e1176c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.715348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.716094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.716688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.716806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.722865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 820/1613] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:29.725150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b2ceb-6403-4562-b8bb-b9b8f8a39b00 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.725857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b2ceb-6403-4562-b8bb-b9b8f8a39b00 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.726444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b2ceb-6403-4562-b8bb-b9b8f8a39b00 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.728844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 821/1614] 10.222.0.39 () {54 vars in 961 bytes} [Wed Jul 13 21:04:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:29.729897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b2ceb-6403-4562-b8bb-b9b8f8a39b00 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.730187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.732936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.745333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b2ceb-6403-4562-b8bb-b9b8f8a39b00 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.746485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 794/1615] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:29.747375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.750952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bea2a0-9b4b-4a08-89a8-c7b5e108a60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.751901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bea2a0-9b4b-4a08-89a8-c7b5e108a60b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.752364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bea2a0-9b4b-4a08-89a8-c7b5e108a60b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.753179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bea2a0-9b4b-4a08-89a8-c7b5e108a60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.758404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c926b-aea5-4bb8-bfdd-07c1058bcb9c service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.765498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f6a7a-f831-477a-8c53-992c11d03175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.766276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6a7a-f831-477a-8c53-992c11d03175 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.766562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6a7a-f831-477a-8c53-992c11d03175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.766860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f6a7a-f831-477a-8c53-992c11d03175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.767183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bea2a0-9b4b-4a08-89a8-c7b5e108a60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.767555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f6a7a-f831-477a-8c53-992c11d03175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.769091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bea2a0-9b4b-4a08-89a8-c7b5e108a60b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.781714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f6a7a-f831-477a-8c53-992c11d03175 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.784140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 822/1616] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:29.786115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77400363-940c-479b-b846-5ecefa0df6ec None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.786570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77400363-940c-479b-b846-5ecefa0df6ec None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.787071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77400363-940c-479b-b846-5ecefa0df6ec None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.788736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 823/1617] 10.222.0.39 () {54 vars in 961 bytes} [Wed Jul 13 21:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:29.789662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77400363-940c-479b-b846-5ecefa0df6ec None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:29.807742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77400363-940c-479b-b846-5ecefa0df6ec None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:29.816079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf327c3-47f4-4206-a820-236f15556d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:29.817194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf327c3-47f4-4206-a820-236f15556d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.818062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf327c3-47f4-4206-a820-236f15556d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.818598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf327c3-47f4-4206-a820-236f15556d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.821142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf327c3-47f4-4206-a820-236f15556d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:29.823724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf327c3-47f4-4206-a820-236f15556d65 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:29.826493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 795/1618] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:29.827972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.828267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.828663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.853079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:29.894728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 824/1619] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:29.898823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.899091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:29.899387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:29.899672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:29.900998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:29.946712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:29.949627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:30.064959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 796/1620] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:30.066537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.107975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:30.127003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 825/1621] 10.222.0.39 () {58 vars in 1058 bytes} [Wed Jul 13 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:30.129684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.166938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701f611-ea7b-48c9-98d3-dc87d29237be None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.173597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.174531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.174996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.175379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.176218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.179454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.210696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 797/1622] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:30 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:30.212273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.237161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1488cb5-6ea3-4d4a-8223-fd4dc5bcf3f2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.253674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.254456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.254870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.255158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.284486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348045f-52ef-4e7f-9518-35d5f6d104b1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.292964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.294759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.295332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.295332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.297634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.301454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.346539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 826/1623] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:30 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:30.347746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.382920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.384573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.390555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb88f2c-dbdd-4a20-b571-229edb864267 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.405458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.406479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.407975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.409230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.413887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 798/1624] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:30.414949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.449882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eaef1-1403-445c-91d7-42c94bb148bf service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.468148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.468845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.469172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.469448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.470190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.476558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.515382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 799/1625] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:30.515382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.531164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb7e30-589b-410d-b512-2c9fbe09017d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.542987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.543681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.544044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.544329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.545070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.551369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.554208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.559168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.561060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:30.576710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 800/1626] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:30.577268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 827/1627] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:30.580597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.591033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.615110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7582b-7766-4fa5-bdd1-e98d07ad26d9 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.624556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c93e3-4d7f-4d8c-85af-592d62e83bae service nova] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.631575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.632589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.632883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.633166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] PATH_INFO: `/v3/projects/a866385d0d974296a441d5a6d43bec0c/users/715b74d8fc0846409619e0907763a660/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.633727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.635347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.636954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.637429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] PATH_INFO: `/v3/projects/cd87b5da894c4269a6d075575f942247/users/3f15ea5d5e4341808a0f2b19f924ee6e/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.670209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd87b5da894c4269a6d075575f942247, user_id=3f15ea5d5e4341808a0f2b19f924ee6e, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.676611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.689078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a866385d0d974296a441d5a6d43bec0c, user_id=715b74d8fc0846409619e0907763a660, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.691931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.702807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 828/1628] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:30 2022] PUT /identity/v3/projects/cd87b5da894c4269a6d075575f942247/users/3f15ea5d5e4341808a0f2b19f924ee6e/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:30.709089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.724455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 801/1629] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:30 2022] PUT /identity/v3/projects/a866385d0d974296a441d5a6d43bec0c/users/715b74d8fc0846409619e0907763a660/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:30.725784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.868429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfe2d9-d592-4100-ab87-63d3cec791f4 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.879636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.881180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.881180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.881180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.882158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.883842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.893119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69920194-2e2d-42cc-a22a-5b912e1caefe admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.908063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.909464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 802/1630] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:04:30.910214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.910771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.911228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.911665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.912797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.914808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.929218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a42c9-2471-42a2-9acb-68bf130674f3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.942494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:30.944952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868e74f-5798-482e-858f-64032220ffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:30.945914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868e74f-5798-482e-858f-64032220ffcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.946199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868e74f-5798-482e-858f-64032220ffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.946477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868e74f-5798-482e-858f-64032220ffcb admin admin] PATH_INFO: `/v3/projects/a866385d0d974296a441d5a6d43bec0c/users/715b74d8fc0846409619e0907763a660/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:30.950691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868e74f-5798-482e-858f-64032220ffcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a866385d0d974296a441d5a6d43bec0c, user_id=715b74d8fc0846409619e0907763a660, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:30.953491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868e74f-5798-482e-858f-64032220ffcb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:30.963035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 829/1631] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:30.964314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:30.987657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 803/1632] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:30 2022] PUT /identity/v3/projects/a866385d0d974296a441d5a6d43bec0c/users/715b74d8fc0846409619e0907763a660/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:30.988970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1911d6c-fd92-4e6a-9681-0901cd39eed3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:30.997512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6c4f5-aee2-4f3c-9dca-2b3231d797de None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:30.997834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6c4f5-aee2-4f3c-9dca-2b3231d797de None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:30.998075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6c4f5-aee2-4f3c-9dca-2b3231d797de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.047592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-49f6c4f5-aee2-4f3c-9dca-2b3231d797de None None] MFA Rules not processed for user `715b74d8fc0846409619e0907763a660`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:31.050722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-49f6c4f5-aee2-4f3c-9dca-2b3231d797de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:31.139340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc41ae-26b7-4ed7-a1a4-a48511b3787c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:31.139914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc41ae-26b7-4ed7-a1a4-a48511b3787c admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.141181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc41ae-26b7-4ed7-a1a4-a48511b3787c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.141511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc41ae-26b7-4ed7-a1a4-a48511b3787c admin admin] PATH_INFO: `/v3/projects/cd87b5da894c4269a6d075575f942247/users/3f15ea5d5e4341808a0f2b19f924ee6e/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.154775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc41ae-26b7-4ed7-a1a4-a48511b3787c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd87b5da894c4269a6d075575f942247, user_id=3f15ea5d5e4341808a0f2b19f924ee6e, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:31.163710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc41ae-26b7-4ed7-a1a4-a48511b3787c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:31.208267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 830/1633] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:30 2022] PUT /identity/v3/projects/cd87b5da894c4269a6d075575f942247/users/3f15ea5d5e4341808a0f2b19f924ee6e/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:31.208267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d899a-6d5a-40f7-8bdf-b0bf419de1e5 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.208267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d899a-6d5a-40f7-8bdf-b0bf419de1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.208267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d899a-6d5a-40f7-8bdf-b0bf419de1e5 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.208706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 831/1634] 10.222.0.39 () {54 vars in 912 bytes} [Wed Jul 13 21:04:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:31.214889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0eb1b-02c0-45a8-8899-9a9e3d1b2989 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.215172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0eb1b-02c0-45a8-8899-9a9e3d1b2989 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.215449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0eb1b-02c0-45a8-8899-9a9e3d1b2989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.230003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 804/1635] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:04:31.231487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143115f-63d2-4e14-98de-c94e06ac5f50 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.231851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143115f-63d2-4e14-98de-c94e06ac5f50 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.232701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143115f-63d2-4e14-98de-c94e06ac5f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.253865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c143115f-63d2-4e14-98de-c94e06ac5f50 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:31.268901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c143115f-63d2-4e14-98de-c94e06ac5f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:31.304332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e3e0eb1b-02c0-45a8-8899-9a9e3d1b2989 None None] MFA Rules not processed for user `3f15ea5d5e4341808a0f2b19f924ee6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:31.305557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e3e0eb1b-02c0-45a8-8899-9a9e3d1b2989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:31.349637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:31.422357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 805/1636] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:04:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:31.424551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1058a8-83e2-42ee-8c07-545fdf445f48 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.424845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1058a8-83e2-42ee-8c07-545fdf445f48 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.425184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1058a8-83e2-42ee-8c07-545fdf445f48 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.432855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 806/1637] 10.222.0.39 () {54 vars in 912 bytes} [Wed Jul 13 21:04:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:31.433832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1058a8-83e2-42ee-8c07-545fdf445f48 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:31.502555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 832/1638] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:31.505105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.505429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.505712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.543864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:31.545290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:31.566858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 833/1639] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:04:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:31.567901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:31.575161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1058a8-83e2-42ee-8c07-545fdf445f48 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:31.590826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:31.591504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.591798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.592068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.698717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d5231-9d99-408e-8001-fde4e5c9ef8a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:31.698717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:31.698717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.698717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.699549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.699549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:31.699549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:31.737858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 834/1640] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:31.740535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:31.743133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:31.768794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 807/1641] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:31.819959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:31.833219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30379de-20b7-45e0-b5c2-3d74c5a435cc service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:31.841430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:31.842456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.842940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.843380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.855705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:31.857302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:31.885114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 835/1642] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:31.935982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:31.949773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8d340-04ba-4d2c-88be-f7fc884fa808 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:31.960485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:31.963228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:31.963679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:31.964845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:31.991310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:31.996959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:32.045608 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 808/1643] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:32.957018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:32.969879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925ae58-d9ad-41dc-b20d-2d3ad06f024f service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:32.991863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:32.992287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:32.992573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:32.996356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:33.039701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:33.052070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa747074-c57a-4188-a664-04a5d047bee8 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:33.075177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fe231-2acb-455e-a231-42cef2ac3557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:33.076666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe231-2acb-455e-a231-42cef2ac3557 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:33.079299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe231-2acb-455e-a231-42cef2ac3557 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:33.081371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fe231-2acb-455e-a231-42cef2ac3557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:33.189494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:33.234908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:33.236520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:33.270210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fe231-2acb-455e-a231-42cef2ac3557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:33.272402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fe231-2acb-455e-a231-42cef2ac3557 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:33.275828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 836/1644] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:33.298333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 809/1645] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:04:35.986474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:36.118187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4763420-9bc7-43f6-956a-b43ff10dbb47 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:36.124925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:36.126048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:36.126647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:36.127202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] PATH_INFO: `/v3/users/a5f7d2d0c8cc4a8db378ca436b0b56b2` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:36.130012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5f7d2d0c8cc4a8db378ca436b0b56b2)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:36.144968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: WARNING py.warnings [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:04:36.144968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: warnings.warn(msg) Jul 13 21:04:36.144968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: Jul 13 21:04:36.148373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:36.377123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5f7d2d0c8cc4a8db378ca436b0b56b2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:36.398585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5f7d2d0c8cc4a8db378ca436b0b56b2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:36.452566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5f7d2d0c8cc4a8db378ca436b0b56b2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:36.453259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5f7d2d0c8cc4a8db378ca436b0b56b2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:36.456230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5f7d2d0c8cc4a8db378ca436b0b56b2'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:36.519969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 837/1646] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:04:35 2022] DELETE /identity/v3/users/a5f7d2d0c8cc4a8db378ca436b0b56b2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:36.521532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:36.522079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:36.522154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:36.548375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:36.567794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:36.570449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:36.707640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 810/1647] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:36.814239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:36.816193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:36.966532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1165a8-e77b-476f-a5e6-5e46b83c018c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:36.978116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaaefc-5529-4d1f-aca5-bbfce23f310c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:36.982573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaaefc-5529-4d1f-aca5-bbfce23f310c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:36.984526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaaefc-5529-4d1f-aca5-bbfce23f310c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:36.985017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaaefc-5529-4d1f-aca5-bbfce23f310c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:37.015796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaaefc-5529-4d1f-aca5-bbfce23f310c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:37.018680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaaefc-5529-4d1f-aca5-bbfce23f310c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:37.040159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 838/1648] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:38.088835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:38.091142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:38.109551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8831f19-0cd6-4197-be5d-9821f7101cc9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:38.115980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:38.117385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:38.117814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:38.118331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] PATH_INFO: `/v3/users/cc27492e934840bdbb6dd179306988d6` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:38.120390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc27492e934840bdbb6dd179306988d6)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:38.159230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:38.297562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc27492e934840bdbb6dd179306988d6'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:38.316711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc27492e934840bdbb6dd179306988d6'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:38.334541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc27492e934840bdbb6dd179306988d6'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:38.335341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc27492e934840bdbb6dd179306988d6'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:38.335969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc27492e934840bdbb6dd179306988d6'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:38.362780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 811/1649] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:04:38 2022] DELETE /identity/v3/users/cc27492e934840bdbb6dd179306988d6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:38.364953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:38.366474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:38.367500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:38.387887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:38.407946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:38.410115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:38.518749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 839/1650] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:38.568007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:38.569721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:38.704090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd335ec-1105-4913-9fd8-0f8da3e9401d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:38.744199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177ed9-2bb7-49c3-a09f-93918053202e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:38.745134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177ed9-2bb7-49c3-a09f-93918053202e service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:38.745948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177ed9-2bb7-49c3-a09f-93918053202e service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:38.746538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177ed9-2bb7-49c3-a09f-93918053202e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:38.768908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177ed9-2bb7-49c3-a09f-93918053202e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:38.772537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177ed9-2bb7-49c3-a09f-93918053202e service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:38.789701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 812/1651] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:38.877256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:38.879654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:38.905159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cb5e7-7610-4385-bb21-4206a40c69dc None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:38.916754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:38.919032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:38.919535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:38.920020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] PATH_INFO: `/v3/projects/93560bcf2e774cfa90a37675d206c510` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:38.921855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93560bcf2e774cfa90a37675d206c510)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:38.925336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:39.027710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93560bcf2e774cfa90a37675d206c510', 'request_id': 'req-9f64345d-9cc0-40bb-9017-fe8663738430'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:39.043911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93560bcf2e774cfa90a37675d206c510', 'request_id': 'req-9f64345d-9cc0-40bb-9017-fe8663738430'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:39.062990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93560bcf2e774cfa90a37675d206c510', 'request_id': 'req-9f64345d-9cc0-40bb-9017-fe8663738430'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:39.099472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] The token cache is being invalidate because project 93560bcf2e774cfa90a37675d206c510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:04:39.099996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:39.101401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9f64345d-9cc0-40bb-9017-fe8663738430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:39.107338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 840/1652] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:04:38 2022] DELETE /identity/v3/projects/93560bcf2e774cfa90a37675d206c510 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:39.112129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.113135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.113668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.154203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:39.166891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:39.215119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:39.322657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.323022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.323304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.327304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 813/1653] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:39.384582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:39.389464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:39.428543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:39.437013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 841/1654] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:39.440602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d409f-075d-48ec-826a-679d0b3c9068 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:39.455404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:39.457667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.459670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.472964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.476212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:39.478085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:39.485813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73293a61-7d57-40f5-ae26-79fd609efdf1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:39.514711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 814/1655] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:39.519790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:39.536212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57aaf4-b601-4669-a218-c12460b20084 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:39.552167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:39.553805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.555715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.556573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.557490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:39.580174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:39.612958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.613342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.613697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.659131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 842/1656] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:39 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:39.666521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:39.679760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] MFA Rules not processed for user `fc4afeb23348438eabfe22f1a975f9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:39.685031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:39.814658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 815/1657] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:39.824461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193064f-9aa2-49f1-9766-c3536a1456ec admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:39.830857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:39.832722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:39.840984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:39.841977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.842340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.842745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.843518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:39.851877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:39.888820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:39.919406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 843/1658] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:39 2022] POST /identity/v3/users => generated 287 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:39.923387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:39.944411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a24e3f-2a2d-47d7-8581-63432ff2392c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:39.959148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:39.960062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:39.960214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:39.960573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:39.961245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:39.963050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:39.994362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 844/1659] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:04:39.998358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:40.018220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140104a-1ae0-4271-b0f2-63d3c0b39489 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:40.024480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7873260-06ae-44f5-81ab-b092e64f3775 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:40.029435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:40.030134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.030416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.030713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.031458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:40.033058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:40.039879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e424-5337-41af-88d6-eebb7af78d49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:40.040940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e424-5337-41af-88d6-eebb7af78d49 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.041544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e424-5337-41af-88d6-eebb7af78d49 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.041827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e424-5337-41af-88d6-eebb7af78d49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.052669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 845/1660] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:04:40.056995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:40.060019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8e424-5337-41af-88d6-eebb7af78d49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:40.063560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8e424-5337-41af-88d6-eebb7af78d49 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:40.078405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 816/1661] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:04:40.079810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d00fa-36ac-4ec9-b337-8bd757c124b3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:40.091222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:40.092400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.092689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.092964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] PATH_INFO: `/v3/projects/6908a9ea6abf4cadaa65a2690b525eba/users/63c04214c85e412c80d23fefd41ef4bb/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.118359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6908a9ea6abf4cadaa65a2690b525eba, user_id=63c04214c85e412c80d23fefd41ef4bb, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:40.121285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:40.154342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.154877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.154963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.157855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 846/1662] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:40 2022] PUT /identity/v3/projects/6908a9ea6abf4cadaa65a2690b525eba/users/63c04214c85e412c80d23fefd41ef4bb/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:40.206558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] MFA Rules not processed for user `63c04214c85e412c80d23fefd41ef4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:40.207804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:40.246709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:40.353143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 817/1663] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:40.360157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:40.525620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbeb11a-e6c4-46fa-a37b-a167ec7a99ce admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:40.539968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:40.540670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.540958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.541242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.541965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:40.548208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:40.612203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 847/1664] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:40 2022] POST /identity/v3/projects => generated 373 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:40.616539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:40.632742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d48914-4d24-4cea-a640-ff2b98c6d901 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:40.803521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:40.804249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.804543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.804842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.805584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:40.807609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:40.858031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 818/1665] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:40 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:40.862065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:40.873227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce48b-b9bf-4ff6-b7d0-073cff2d960a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:40.904207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:40.904786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:40.905068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:40.905346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:40.906095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:40.915228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:40.974543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:40.979921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 848/1666] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:40.997285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d12e-8c65-4527-a5a8-35589ca13a93 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:41.009687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:41.010525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:41.010844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:41.011151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] PATH_INFO: `/v3/projects/1ca748a4d3eb478aacf60ce011fa8efa/users/2b63e1bc4fec492f931ecc5d9ea68bf9/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:41.050915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca748a4d3eb478aacf60ce011fa8efa, user_id=2b63e1bc4fec492f931ecc5d9ea68bf9, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:41.054161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:41.085472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 819/1667] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:40 2022] PUT /identity/v3/projects/1ca748a4d3eb478aacf60ce011fa8efa/users/2b63e1bc4fec492f931ecc5d9ea68bf9/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:41.092927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:41.113393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4189-04a0-409e-ae86-38995f2a0977 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:41.284998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:41.285990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:41.286352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:41.287005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:41.289242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:41.294381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:41.332984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 849/1668] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:41.335912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:41.352877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58defa1f-3eb7-4be1-b971-99e9127ed86c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:41.361953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e60368-0425-48f0-b332-4cdf2279b2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:41.362905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e60368-0425-48f0-b332-4cdf2279b2a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:41.363230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e60368-0425-48f0-b332-4cdf2279b2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:41.363516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e60368-0425-48f0-b332-4cdf2279b2a6 admin admin] PATH_INFO: `/v3/projects/1ca748a4d3eb478aacf60ce011fa8efa/users/2b63e1bc4fec492f931ecc5d9ea68bf9/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:41.368459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e60368-0425-48f0-b332-4cdf2279b2a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca748a4d3eb478aacf60ce011fa8efa, user_id=2b63e1bc4fec492f931ecc5d9ea68bf9, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:41.371517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e60368-0425-48f0-b332-4cdf2279b2a6 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:41.402300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 820/1669] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:41 2022] PUT /identity/v3/projects/1ca748a4d3eb478aacf60ce011fa8efa/users/2b63e1bc4fec492f931ecc5d9ea68bf9/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:41.410769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:41.414314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:41.415000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:41.415903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:41.420176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:41.451591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0021d1b-2d94-4b8b-8902-8d3f298102e8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:41.481059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:41.496974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] MFA Rules not processed for user `2b63e1bc4fec492f931ecc5d9ea68bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:41.498228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:41.623137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:41.624638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:41.625058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:41.625467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] PATH_INFO: `/v3/projects/ed45078466534951ba448cfaa895ad1d` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:41.628692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed45078466534951ba448cfaa895ad1d)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:41.630994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:41.637094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:41.694366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 821/1670] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:41.708337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:41.710531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:41.715416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed45078466534951ba448cfaa895ad1d', 'request_id': 'req-271e9261-b529-42ce-bee6-e115bf420674'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:41.722317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed45078466534951ba448cfaa895ad1d', 'request_id': 'req-271e9261-b529-42ce-bee6-e115bf420674'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:41.737988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed45078466534951ba448cfaa895ad1d', 'request_id': 'req-271e9261-b529-42ce-bee6-e115bf420674'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:41.765858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] The token cache is being invalidate because project ed45078466534951ba448cfaa895ad1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:04:41.766194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:41.767055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:41.772778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 850/1671] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:04:41 2022] DELETE /identity/v3/projects/ed45078466534951ba448cfaa895ad1d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:41.777083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:41.780526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:41.840888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303a63-2074-49db-8e04-7de049522ffd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:41.967879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:41.967879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:41.967879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:41.967879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:41.978180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:41.994355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e9261-b529-42ce-bee6-e115bf420674 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:42.036554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:42.037518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:42.037797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:42.039676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] PATH_INFO: `/v3/users/715b74d8fc0846409619e0907763a660` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:42.040388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715b74d8fc0846409619e0907763a660)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:42.042374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:42.133419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:42.134924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:42.170333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 822/1672] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:04:42.197699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715b74d8fc0846409619e0907763a660'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:42.199787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:42.218818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715b74d8fc0846409619e0907763a660'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:42.243173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715b74d8fc0846409619e0907763a660'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:42.244129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715b74d8fc0846409619e0907763a660'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:42.244954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0394d7e6-7e33-40cd-acff-d741d37c3a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715b74d8fc0846409619e0907763a660'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:42.272049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 851/1673] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:04:41 2022] DELETE /identity/v3/users/715b74d8fc0846409619e0907763a660 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:42.369711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:42.371315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:42.396340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50179915-08bc-4bac-9629-9dde7b5f45e0 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:42.542951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:42.543965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:42.544396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:42.544834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:42.552314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:42.584962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:42.636674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:42.637874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:42.638363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:42.741991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] MFA Rules not processed for user `d9cab6f67cd34c46886d036ba170331e`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:42.744455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:42.756671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:42.758148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:42.785868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 823/1674] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:42.968697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 852/1675] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:42.981781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:42.983357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:43.269324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7fd44-61da-4fdc-8749-8be0aaf920bf service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:43.295838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30928480-9484-4473-87e7-0a3744de9916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:43.296591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30928480-9484-4473-87e7-0a3744de9916 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:43.297067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30928480-9484-4473-87e7-0a3744de9916 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:43.297366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30928480-9484-4473-87e7-0a3744de9916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:43.319771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30928480-9484-4473-87e7-0a3744de9916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:43.321862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30928480-9484-4473-87e7-0a3744de9916 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:43.339952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 824/1676] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:44.641760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:44.644322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:44.839904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b084a4-f2a8-4c0d-855f-d01303cee569 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:44.855134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:44.856503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:44.858252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:44.858915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] PATH_INFO: `/v3/projects/a866385d0d974296a441d5a6d43bec0c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:44.863458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a866385d0d974296a441d5a6d43bec0c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:44.867526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:44.987016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a866385d0d974296a441d5a6d43bec0c', 'request_id': 'req-10184df9-417d-4641-abe3-31c4c690f587'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:45.001813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a866385d0d974296a441d5a6d43bec0c', 'request_id': 'req-10184df9-417d-4641-abe3-31c4c690f587'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:45.007911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:INSERT=1,keystone:DELETE=8,keystone:SELECT=91 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:45.018090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a866385d0d974296a441d5a6d43bec0c', 'request_id': 'req-10184df9-417d-4641-abe3-31c4c690f587'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:45.049843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] The token cache is being invalidate because project a866385d0d974296a441d5a6d43bec0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:04:45.050175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:45.050831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-10184df9-417d-4641-abe3-31c4c690f587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:04:45.052873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 853/1677] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:04:44 2022] DELETE /identity/v3/projects/a866385d0d974296a441d5a6d43bec0c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:45.057547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.058477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.059006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.127756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:45.132201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:45.310723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 825/1678] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:45.318966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.319253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.319530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.353650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:45.361290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:45.380078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 854/1679] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:45.384856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:45.398370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3bb93-1fc1-443e-b838-8a25d330c246 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:45.417588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:45.418568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.419834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.420269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.423403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:04:45.425572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:45.429694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:45.448158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 826/1680] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:04:45.452108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:45.465683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca53794-e2d1-4260-b5d8-6d40e8a0a46d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:45.477205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:45.478320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.478814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.482592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.483725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:45.489039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:45.564087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 855/1681] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:45 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:45.566007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:45.586281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcc176-b58e-4e7e-bb15-17090c154d36 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:45.648860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:45.715866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:45.716572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.716863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.717147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.717864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:45.719874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:45.767804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 827/1682] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:45 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:45.771870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:45.788772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9156b-3508-44b1-a4eb-a530e616ec31 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:45.808179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:45.809273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.809826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.810218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.811646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:45.813423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:45.841769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:45.842894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 856/1683] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:45.854008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df64ddb-2d33-482d-a4b8-d226e708b73e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:45.864736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1980343-c9e4-4f1f-bc64-5494f10a0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:45.866101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1980343-c9e4-4f1f-bc64-5494f10a0f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.866389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1980343-c9e4-4f1f-bc64-5494f10a0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.866788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1980343-c9e4-4f1f-bc64-5494f10a0f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.868138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1980343-c9e4-4f1f-bc64-5494f10a0f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:45.869797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1980343-c9e4-4f1f-bc64-5494f10a0f32 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:45.898175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 828/1684] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:45.903417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:45.924584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1f188-64fb-4a88-8a1f-5408f9cfc168 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:45.943575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e2d2f-4a1a-427a-b10e-ba0364562ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:45.944637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e2d2f-4a1a-427a-b10e-ba0364562ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:45.944936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e2d2f-4a1a-427a-b10e-ba0364562ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:45.945221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e2d2f-4a1a-427a-b10e-ba0364562ed6 admin admin] PATH_INFO: `/v3/projects/c950aefee69f491eb643371682266e12/users/9a5a1d934b1d42b2bf195b7c6183c632/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:45.989483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e2d2f-4a1a-427a-b10e-ba0364562ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c950aefee69f491eb643371682266e12, user_id=9a5a1d934b1d42b2bf195b7c6183c632, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:45.993182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e2d2f-4a1a-427a-b10e-ba0364562ed6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:46.023452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.024085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.024566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:46.095550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 857/1685] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:45 2022] PUT /identity/v3/projects/c950aefee69f491eb643371682266e12/users/9a5a1d934b1d42b2bf195b7c6183c632/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:46.097193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce7e1-b5f0-450c-942e-42fb0126f64b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.097491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce7e1-b5f0-450c-942e-42fb0126f64b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.110584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce7e1-b5f0-450c-942e-42fb0126f64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:46.129798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] MFA Rules not processed for user `a28051d0f956461c8f3b39825e33af96`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:46.132171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:46.186490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f77ce7e1-b5f0-450c-942e-42fb0126f64b None None] MFA Rules not processed for user `9a5a1d934b1d42b2bf195b7c6183c632`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:46.190698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f77ce7e1-b5f0-450c-942e-42fb0126f64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:46.394236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 858/1686] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:46.395842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5240a-9957-4de9-af9d-27aa04b6e1f5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.396715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5240a-9957-4de9-af9d-27aa04b6e1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.397096 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5240a-9957-4de9-af9d-27aa04b6e1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:46.437610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5bf5240a-9957-4de9-af9d-27aa04b6e1f5 None None] MFA Rules not processed for user `fc4afeb23348438eabfe22f1a975f9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:46.439763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5bf5240a-9957-4de9-af9d-27aa04b6e1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:46.444114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 829/1687] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:04:46.445226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:46.562499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 859/1688] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:46.564430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.564728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.565013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:46.618631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a72e5a-4466-4826-aa53-0d5e359f70c8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:46.636686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:46.637769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.638980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.640611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:46.641384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:46.647286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:46.667174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] MFA Rules not processed for user `a28051d0f956461c8f3b39825e33af96`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:46.669855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:46.698254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 860/1689] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:46.711941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:46.714544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:46.719346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 830/1690] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:04:46 2022] POST /identity/v3/projects => generated 375 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:46.722090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:46.755228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:46.793856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:46.883826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de250da-148c-4769-8367-ef55a6936c4b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:46.967215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc94cd2-18a5-40d6-830a-2f747694559d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:46.984087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:46.984967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.985262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.985751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:46.987150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:46.994127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:46.996126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:46.996943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:46.997300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:46.997647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.061395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 831/1691] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:04:46 2022] POST /identity/v3/users => generated 294 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:04:47.065563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:47.088014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28309-53cf-4ad1-a60f-77f97c6615db admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:47.098362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c9fc3-8288-4b95-aee0-8d3fc0164082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:47.099016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c9fc3-8288-4b95-aee0-8d3fc0164082 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.099337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c9fc3-8288-4b95-aee0-8d3fc0164082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.101714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c9fc3-8288-4b95-aee0-8d3fc0164082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.102462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c9fc3-8288-4b95-aee0-8d3fc0164082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:47.104224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c9fc3-8288-4b95-aee0-8d3fc0164082 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:47.128269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:47.130362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 832/1692] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:47.132468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d71b4b-9487-466c-a1d9-99a69527e18c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.132717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d71b4b-9487-466c-a1d9-99a69527e18c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.133605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:47.137290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d71b4b-9487-466c-a1d9-99a69527e18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.145052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 861/1693] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:47.145962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:47.158381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b95eb-be63-4929-8b27-5094d5afed1f service nova] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:47.163228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:47.164135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.164421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.164753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] PATH_INFO: `/v3/projects/a5b0be1610ef4763aaba135fb62c4229/users/ca1ca412351148e1865196515525a295/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.184093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0be1610ef4763aaba135fb62c4229, user_id=ca1ca412351148e1865196515525a295, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:47.186875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:47.219883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 862/1694] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:47 2022] PUT /identity/v3/projects/a5b0be1610ef4763aaba135fb62c4229/users/ca1ca412351148e1865196515525a295/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:47.224977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:47.237131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-65d71b4b-9487-466c-a1d9-99a69527e18c None None] MFA Rules not processed for user `0884a33a47e448e7826f257880a28964`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:47.238387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-65d71b4b-9487-466c-a1d9-99a69527e18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:47.275066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:47.369048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776dffc-82c2-448e-814a-2579abbadb61 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:47.375220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:47.376124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.376660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.377085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.377967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:47.381528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:47.397047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 863/1695] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:04:47.401806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:47.430559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c5ead-4175-473f-a4f5-e48ad8b8ed7f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:47.430981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 833/1696] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:47.449666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:47.450659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.450953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.451283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] PATH_INFO: `/v3/projects/a5b0be1610ef4763aaba135fb62c4229/users/ca1ca412351148e1865196515525a295/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.455632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0be1610ef4763aaba135fb62c4229, user_id=ca1ca412351148e1865196515525a295, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:47.459673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:47.486398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 864/1697] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:04:47 2022] PUT /identity/v3/projects/a5b0be1610ef4763aaba135fb62c4229/users/ca1ca412351148e1865196515525a295/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:04:47.499777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba39ec-e0d4-41d4-b82a-a735634553fa None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.500174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba39ec-e0d4-41d4-b82a-a735634553fa None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.500821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba39ec-e0d4-41d4-b82a-a735634553fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.555672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0cba39ec-e0d4-41d4-b82a-a735634553fa None None] MFA Rules not processed for user `ca1ca412351148e1865196515525a295`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:47.557301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0cba39ec-e0d4-41d4-b82a-a735634553fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:47.723443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 834/1698] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:47.732285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:47.734304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:47.825242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f141e08-bea6-4de3-a122-c114198e65a9 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.825398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f141e08-bea6-4de3-a122-c114198e65a9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.825672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f141e08-bea6-4de3-a122-c114198e65a9 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.827106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 835/1699] 10.222.0.39 () {54 vars in 912 bytes} [Wed Jul 13 21:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:47.832840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.833132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.833412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.888308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:47.893763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ac97a-c119-4de3-a019-1fbe3b45e147 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:47.910091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0f0e0-fda8-4e50-8780-fe57164f1d77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:47.910864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f0e0-fda8-4e50-8780-fe57164f1d77 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:47.915854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f0e0-fda8-4e50-8780-fe57164f1d77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:47.916616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0f0e0-fda8-4e50-8780-fe57164f1d77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:47.947711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:47.949023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:48.061421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0f0e0-fda8-4e50-8780-fe57164f1d77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:48.067463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0f0e0-fda8-4e50-8780-fe57164f1d77 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:48.100099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 865/1700] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:48.101475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 836/1701] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:48.102512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:48.104164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:48.104841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c62ee-be8c-45b5-bfa0-b22aa0f8589a None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:48.106089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c62ee-be8c-45b5-bfa0-b22aa0f8589a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:48.106381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c62ee-be8c-45b5-bfa0-b22aa0f8589a None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:48.118322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 866/1702] 10.222.0.39 () {54 vars in 912 bytes} [Wed Jul 13 21:04:48 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:48.122838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3113b26-3c53-4372-bbb2-aac60666b45d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:48.124086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3113b26-3c53-4372-bbb2-aac60666b45d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:48.124179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44906-8955-47bf-b508-c5b421af63e9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:48.124253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3113b26-3c53-4372-bbb2-aac60666b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:48.137749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:48.140523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:48.140744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:48.141051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:48.161644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:48.163281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:48.167387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e3113b26-3c53-4372-bbb2-aac60666b45d None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:48.177602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e3113b26-3c53-4372-bbb2-aac60666b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:48.185837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 867/1703] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:48.187903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5295a5-51ba-40d4-8e1e-fb83f5830423 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:48.188268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5295a5-51ba-40d4-8e1e-fb83f5830423 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:48.188670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5295a5-51ba-40d4-8e1e-fb83f5830423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:48.215347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 837/1704] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:48.229003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:48.230416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1d5295a5-51ba-40d4-8e1e-fb83f5830423 None None] MFA Rules not processed for user `d9cab6f67cd34c46886d036ba170331e`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:48.231117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:48.233495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1d5295a5-51ba-40d4-8e1e-fb83f5830423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:48.263679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d105352-6bf5-442f-8dd3-3b3fe6f893e3 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:48.280549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:48.284023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:48.285769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:48.291770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:48.319727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:48.321197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:48.350835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:48.351737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 838/1705] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:48.389771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 868/1706] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:49.093705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:49.247684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a134994-c8fd-49ff-b3d8-00c3cb7b0898 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:49.248162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a134994-c8fd-49ff-b3d8-00c3cb7b0898 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:49.249542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a134994-c8fd-49ff-b3d8-00c3cb7b0898 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:49.251756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 869/1707] 10.222.0.39 () {54 vars in 960 bytes} [Wed Jul 13 21:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:04:49.261920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1ab49-b7b8-4a45-85c7-0c1466ed16e9 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:49.274187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:49.275004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:49.275729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:49.276090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:49.276885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:49.277163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:49.277441 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:49.324063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:49.341662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:49.445093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:49.448198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:49.505055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 839/1708] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:49.507350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:49.515350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:49.547664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 870/1709] 10.222.0.39 () {58 vars in 1057 bytes} [Wed Jul 13 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:49.557015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:49.570026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d678-89f2-4d99-8897-4877274c1667 service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:49.575700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:49.576459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:49.576840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:49.577250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:49.579744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186572b-2b57-49bf-95df-4a4c873631c9 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:49.581502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:49.591842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06513401-c18a-46f5-b629-e081b0391237 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:49.592783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06513401-c18a-46f5-b629-e081b0391237 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:49.594215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06513401-c18a-46f5-b629-e081b0391237 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:49.595126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06513401-c18a-46f5-b629-e081b0391237 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:49.618050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:49.619948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2cc90-d971-4402-a30a-a343a319ede9 service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:49.631976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06513401-c18a-46f5-b629-e081b0391237 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:49.634658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06513401-c18a-46f5-b629-e081b0391237 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:49.658890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 840/1710] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:04:49.659728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 871/1711] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:04:52.241886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:52.242441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:52.242441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:52.249726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:04:52.262592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] MFA Rules not processed for user `0884a33a47e448e7826f257880a28964`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:52.263849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:52.295914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 841/1712] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:04:59.275840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:59.276717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:59.277237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:59.361257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:04:59.363155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:59.460795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 872/1713] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:04:59.691568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:59.694144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:59.852281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:04:59.854689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:04:59.884634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac3b3a-0be1-42a2-b3c7-4647af7c8d2a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:59.895530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:59.897676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:59.898644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:59.899457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:59.922062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:59.924099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:59.944382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb94d5-442e-4d31-810e-9fd335701126 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:04:59.953597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f2de-6c91-4b62-a061-d74249c297e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:04:59.954463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f2de-6c91-4b62-a061-d74249c297e4 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:04:59.954795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f2de-6c91-4b62-a061-d74249c297e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:04:59.955203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f2de-6c91-4b62-a061-d74249c297e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:04:59.968183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 873/1714] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:04:59.973449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93f2de-6c91-4b62-a061-d74249c297e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:04:59.976746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93f2de-6c91-4b62-a061-d74249c297e4 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:04:59.992259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 842/1715] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:04:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:05:01.152144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:01.153960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.205079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.205079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.205079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.234574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:01.238767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.255587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 843/1716] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Jul 13 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:01.278493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:01.327443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614c20-ab42-454a-ba6a-8812433ecb18 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:01.337827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:01.339256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.340040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.340430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] PATH_INFO: `/v3/users/b4b83f5475f04e6d8435eae9da442d43` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.342910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b83f5475f04e6d8435eae9da442d43)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:01.345255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:01.390424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:01.393658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.417773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52276861-f20d-4b51-9912-be917c1b7a99 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:01.422819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:01.423824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.424093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.424597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] PATH_INFO: `/v3/users/4711f2212b2b42b1bfe414d2795c6356` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.426435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4711f2212b2b42b1bfe414d2795c6356)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:01.430282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:01.470402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b83f5475f04e6d8435eae9da442d43'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.486515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b83f5475f04e6d8435eae9da442d43'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.545064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4711f2212b2b42b1bfe414d2795c6356'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.559004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b83f5475f04e6d8435eae9da442d43'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.560768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b83f5475f04e6d8435eae9da442d43'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.561591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-fc267ed1-0e7d-4644-906b-562a47a9aed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b83f5475f04e6d8435eae9da442d43'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.565013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4711f2212b2b42b1bfe414d2795c6356'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.606011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 874/1717] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:01 2022] DELETE /identity/v3/users/b4b83f5475f04e6d8435eae9da442d43 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:01.606168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4711f2212b2b42b1bfe414d2795c6356'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.606717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4711f2212b2b42b1bfe414d2795c6356'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.607495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4d5c06be-35c8-4f28-b2c8-afb2024df04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4711f2212b2b42b1bfe414d2795c6356'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:01.611075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8127b07-14c2-4800-80e9-284234661034 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.611500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8127b07-14c2-4800-80e9-284234661034 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.611997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8127b07-14c2-4800-80e9-284234661034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.627581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 844/1718] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:01 2022] DELETE /identity/v3/users/4711f2212b2b42b1bfe414d2795c6356 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:01.632265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.632764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.633185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.634041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a8127b07-14c2-4800-80e9-284234661034 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:01.639809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a8127b07-14c2-4800-80e9-284234661034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.668226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:01.670041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.742417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:01.753566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 875/1719] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:01.770382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8127b07-14c2-4800-80e9-284234661034 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:01.778157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a8127b07-14c2-4800-80e9-284234661034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.833968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 845/1720] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:01.851096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:01.852581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:01.922558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8127b07-14c2-4800-80e9-284234661034 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:01.930554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:01.931246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.931527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.931804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.953377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:01.959724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeb3a8-9022-42cc-9e0c-3efc9d2ae44a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:01.960940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:01.967685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:01.968656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:01.969468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:01.970477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:01.976134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:01.985999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:01.988124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:01.992410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 876/1721] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:02.043203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 846/1722] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:02.706978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:02.715071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:02.741312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31573f73-e6c9-4a25-989e-2432c191d898 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:02.755797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:02.757237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:02.757700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:02.758071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] PATH_INFO: `/v3/projects/95155d588e674097b7b73aad68e3c651` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:02.760295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95155d588e674097b7b73aad68e3c651)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:02.763234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:02.775636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:02.777223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:02.800589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276facb5-a22b-4ebc-9938-1396c746071e service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:02.811273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:02.812670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:02.813124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:02.813562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] PATH_INFO: `/v3/projects/c317c7bdea394bcdbc0657fadedac2f7` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:02.815033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c317c7bdea394bcdbc0657fadedac2f7)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:02.816422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: WARNING py.warnings [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 21:05:02.816422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: warnings.warn(msg) Jul 13 21:05:02.816422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: Jul 13 21:05:02.818472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:02.843127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95155d588e674097b7b73aad68e3c651', 'request_id': 'req-5afff303-6246-4c8e-8a04-3b1123680a1b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.850794 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95155d588e674097b7b73aad68e3c651', 'request_id': 'req-5afff303-6246-4c8e-8a04-3b1123680a1b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.861135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95155d588e674097b7b73aad68e3c651', 'request_id': 'req-5afff303-6246-4c8e-8a04-3b1123680a1b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.875509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] The token cache is being invalidate because project 95155d588e674097b7b73aad68e3c651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:02.875796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.876430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.877983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 877/1723] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:02 2022] DELETE /identity/v3/projects/95155d588e674097b7b73aad68e3c651 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:02.882593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:02.883921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:02.888131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c317c7bdea394bcdbc0657fadedac2f7', 'request_id': 'req-89b82f94-d640-44a4-8214-94b7a483eb60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.902631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c317c7bdea394bcdbc0657fadedac2f7', 'request_id': 'req-89b82f94-d640-44a4-8214-94b7a483eb60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.908587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c317c7bdea394bcdbc0657fadedac2f7', 'request_id': 'req-89b82f94-d640-44a4-8214-94b7a483eb60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.921682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] The token cache is being invalidate because project c317c7bdea394bcdbc0657fadedac2f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:02.921971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.922655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-89b82f94-d640-44a4-8214-94b7a483eb60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:02.924206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 847/1724] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:02 2022] DELETE /identity/v3/projects/c317c7bdea394bcdbc0657fadedac2f7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:02.929382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e28e05-7c6e-4169-a19d-57ae942472d4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:02.929667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e28e05-7c6e-4169-a19d-57ae942472d4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:02.929978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e28e05-7c6e-4169-a19d-57ae942472d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:02.949853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c8e28e05-7c6e-4169-a19d-57ae942472d4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:02.951247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c8e28e05-7c6e-4169-a19d-57ae942472d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:02.955256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afff303-6246-4c8e-8a04-3b1123680a1b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.002562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:03.011652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.012695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.013105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.013385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] PATH_INFO: `/v3/users/3f15ea5d5e4341808a0f2b19f924ee6e` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.019987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 848/1725] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:03.020149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f15ea5d5e4341808a0f2b19f924ee6e)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.020149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.021431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.021752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.022036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.041150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:03.042390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.051512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:03.052693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 849/1726] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:03.056983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.067479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f15ea5d5e4341808a0f2b19f924ee6e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.075537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789df91-f350-4fa0-ae6c-de7877e02609 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.077755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f15ea5d5e4341808a0f2b19f924ee6e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.082012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.082721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.083041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.083320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.084202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:03.084645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.086392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.092293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f15ea5d5e4341808a0f2b19f924ee6e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.093100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f15ea5d5e4341808a0f2b19f924ee6e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.094002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f15ea5d5e4341808a0f2b19f924ee6e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.099845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 850/1727] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:03.104370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.106533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.110744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 878/1728] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:02 2022] DELETE /identity/v3/users/3f15ea5d5e4341808a0f2b19f924ee6e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:03.122924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7039b-6bfb-4990-87d8-b983b389e6b7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.201795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.202576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.202967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.203312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.204224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.206233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.243230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 851/1729] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:03.244562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.256042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e76b0-7b7e-4376-a816-1386a827afe6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.338205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.338919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.339208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.339486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.340181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.342146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.363216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 879/1730] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:03.367912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.378459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bea7b-354c-4cb0-bb82-6e7a29f0c721 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.383149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.383916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.384096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.384429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.385292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.387015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.398463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 852/1731] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:03.402795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.408817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.411885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.416011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f89c5c-ad80-4319-90f7-d4c8c150e2a8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.421261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.422097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.422279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.422605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.423437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.425448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.432323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8777d-713b-4cc1-82eb-2707f3751a0b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.440340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 880/1732] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:05:03.440833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.444827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.445822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.446110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.446416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] PATH_INFO: `/v3/projects/cd87b5da894c4269a6d075575f942247` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.447665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd87b5da894c4269a6d075575f942247)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.449572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.451029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac135a-b4f3-48f7-86f2-e3a973a307cd admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.455557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623aa62-7f00-48f6-a0e9-b95e3f4a5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.456429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623aa62-7f00-48f6-a0e9-b95e3f4a5990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.456717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623aa62-7f00-48f6-a0e9-b95e3f4a5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.456995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623aa62-7f00-48f6-a0e9-b95e3f4a5990 admin admin] PATH_INFO: `/v3/projects/e491c88c32184fcc9dc795069bcc5a64/users/e48bbad6334e4e32a652297a6196ebe8/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.474496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623aa62-7f00-48f6-a0e9-b95e3f4a5990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e491c88c32184fcc9dc795069bcc5a64, user_id=e48bbad6334e4e32a652297a6196ebe8, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.477506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623aa62-7f00-48f6-a0e9-b95e3f4a5990 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.491188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 881/1733] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:03 2022] PUT /identity/v3/projects/e491c88c32184fcc9dc795069bcc5a64/users/e48bbad6334e4e32a652297a6196ebe8/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:03.498999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd87b5da894c4269a6d075575f942247', 'request_id': 'req-f66f2360-6931-470c-b128-f381ce68b0d0'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.502190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.502473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.502786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.506890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd87b5da894c4269a6d075575f942247', 'request_id': 'req-f66f2360-6931-470c-b128-f381ce68b0d0'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.512938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd87b5da894c4269a6d075575f942247', 'request_id': 'req-f66f2360-6931-470c-b128-f381ce68b0d0'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.526552 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] The token cache is being invalidate because project cd87b5da894c4269a6d075575f942247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:03.526871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.527512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f66f2360-6931-470c-b128-f381ce68b0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:03.529107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 853/1734] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:03 2022] DELETE /identity/v3/projects/cd87b5da894c4269a6d075575f942247 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:03.529683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] MFA Rules not processed for user `e48bbad6334e4e32a652297a6196ebe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:03.530924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.533926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db71243-0e6b-4fbb-8bc3-c4e7b7692d3f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.534214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db71243-0e6b-4fbb-8bc3-c4e7b7692d3f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.534494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db71243-0e6b-4fbb-8bc3-c4e7b7692d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.563197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5db71243-0e6b-4fbb-8bc3-c4e7b7692d3f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:03.564451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5db71243-0e6b-4fbb-8bc3-c4e7b7692d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.603732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:03.626756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 882/1735] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:03.629012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:03.631274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.633013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.673354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 854/1736] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:03.683774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.684495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.685616 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.705852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:03.707287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:03.725869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 855/1737] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:03.730065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.732225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a879e-bcd8-4674-b626-e3dd15f03830 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.738843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.739154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67febe-f9c8-4f22-8346-ecc4fa4ee124 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.739565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.740207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.740498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.741311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.743694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.743879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.744541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.744824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.745258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.745839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:03.746143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.747801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.767103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 856/1738] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:05:03.768227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.779758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 883/1739] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:03.781223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.787504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7c297-8331-4ad3-b547-8ad887c6b05a admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.847893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8140a53-dec4-4c99-bdea-1cbde03af074 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.852506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.853175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.853457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.853730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.854419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.855706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.856362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.856635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.856935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.857199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.858082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.860036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.890656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 884/1740] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:03.894379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.905938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901cfd4-f8a4-4e7c-93ff-17aec6f33666 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.909445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 857/1741] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:03.912027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.913035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.913321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.913612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.913728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.914465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.916124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:03.927486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 885/1742] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:03.931299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:03.941477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cdce86-b66b-45ee-be93-d032c9d5e1ad admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.983246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ade205-d17b-4461-8988-36caeb88deab admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:03.994451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:03.995468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:03.995468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:03.995779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:03.996415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:03.999073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.014581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:04.024373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 858/1743] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:03 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:04.024555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.026113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.026490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.027535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] PATH_INFO: `/v3/projects/1fdc64189cd446569b71a9294a55f7eb/users/b719aae0e4ae48598eef4b978f134fb0/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.032332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.046241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cace5d6-acc1-47c4-bf11-4284ccab7302 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.048359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdc64189cd446569b71a9294a55f7eb, user_id=b719aae0e4ae48598eef4b978f134fb0, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.050844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.051315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.051507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.051826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.052109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.052838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.054648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.065209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 859/1744] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:05:04.065757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 886/1745] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:03 2022] PUT /identity/v3/projects/1fdc64189cd446569b71a9294a55f7eb/users/b719aae0e4ae48598eef4b978f134fb0/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:04.070076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.070907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.080206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb85ad-f986-4463-b4d7-5935e9b93a2d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.099465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:04.135128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aef73-144d-4084-89af-fdb0250e9f6c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.143275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.143972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.144263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.144576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.144984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.145326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.145688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.145974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.146248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.146998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.147104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.148965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.157944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 887/1746] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:04.163759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 860/1747] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:04.163903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.164703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.172622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b6c74-1db6-4920-a672-07ce453fa316 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.173991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451cd3b4-3ed8-47f3-9d28-5194a3f2dc0b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.177693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690587e-8261-47d2-ac6b-3d4339acf613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.178552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690587e-8261-47d2-ac6b-3d4339acf613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.178884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690587e-8261-47d2-ac6b-3d4339acf613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.178975 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896dffa-4783-4a70-ba1e-97f77e1541b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.179145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690587e-8261-47d2-ac6b-3d4339acf613 admin admin] PATH_INFO: `/v3/projects/1fdc64189cd446569b71a9294a55f7eb/users/b719aae0e4ae48598eef4b978f134fb0/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.179777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896dffa-4783-4a70-ba1e-97f77e1541b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.180061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896dffa-4783-4a70-ba1e-97f77e1541b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.180332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896dffa-4783-4a70-ba1e-97f77e1541b9 admin admin] PATH_INFO: `/v3/projects/5f5c8c694e154b79af603fb6a8e620b5/users/d86691d8d4684eeabe20fe0213a98392/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.181197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690587e-8261-47d2-ac6b-3d4339acf613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdc64189cd446569b71a9294a55f7eb, user_id=b719aae0e4ae48598eef4b978f134fb0, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.183862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690587e-8261-47d2-ac6b-3d4339acf613 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.194708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 888/1748] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:04 2022] PUT /identity/v3/projects/1fdc64189cd446569b71a9294a55f7eb/users/b719aae0e4ae48598eef4b978f134fb0/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:04.195319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896dffa-4783-4a70-ba1e-97f77e1541b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5c8c694e154b79af603fb6a8e620b5, user_id=d86691d8d4684eeabe20fe0213a98392, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.197962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896dffa-4783-4a70-ba1e-97f77e1541b9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.204898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.205191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.205469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.214452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 861/1749] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:04 2022] PUT /identity/v3/projects/5f5c8c694e154b79af603fb6a8e620b5/users/d86691d8d4684eeabe20fe0213a98392/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:04.225258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.225559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.225842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.232737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] MFA Rules not processed for user `b719aae0e4ae48598eef4b978f134fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:04.233996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:04.251903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] MFA Rules not processed for user `d86691d8d4684eeabe20fe0213a98392`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:04.253096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:04.303461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 889/1750] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:04.314533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.316258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:04.330280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 862/1751] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:04.331292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.389425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:04.394677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c1dc-755e-446c-a644-0c6a08336cc3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.402872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.403546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.404361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.404361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.404779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf5742-2e27-4c7f-9786-11212826a5ae None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.409406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.410143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.410645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.410869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.411692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.413736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.414513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.416057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.426316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 890/1752] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:05:04.442114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 863/1753] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:04 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:04.447321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.462140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979dc3e-bbab-42b5-b31c-d67048093ea6 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.526783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.527463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.527921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.528161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.528851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.530826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.554913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 891/1754] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:04 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:04.556471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.566726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ea8dc-f1d2-43db-b3de-9b892802c74f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.571497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.572208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.572500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.572776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.573501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.575129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.577247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:04.594560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 864/1755] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:04.595578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.606107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39177f-ef10-412d-a58f-f8706ccc294c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.610722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.611607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.611898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.612180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] PATH_INFO: `/v3/projects/d4068941ed944431b1c2f8c129f56e73/users/3cf0d099461a433bad6719cbe7051078/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.632121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4068941ed944431b1c2f8c129f56e73, user_id=3cf0d099461a433bad6719cbe7051078, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.634915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.648607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 892/1756] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:04 2022] PUT /identity/v3/projects/d4068941ed944431b1c2f8c129f56e73/users/3cf0d099461a433bad6719cbe7051078/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:04.652801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.663413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb67180-08f8-4a2c-b614-f7ad05a4339e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.722935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90900769-b134-419d-93f5-d32878c34e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.724187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900769-b134-419d-93f5-d32878c34e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.724489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900769-b134-419d-93f5-d32878c34e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.724775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900769-b134-419d-93f5-d32878c34e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.725520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90900769-b134-419d-93f5-d32878c34e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.727168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90900769-b134-419d-93f5-d32878c34e16 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.741326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 865/1757] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:04.742666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:04.754815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3bee-78f6-4523-88d3-67c4a5bf03e9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:04.764712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:04.765614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.765901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.766176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] PATH_INFO: `/v3/projects/d4068941ed944431b1c2f8c129f56e73/users/3cf0d099461a433bad6719cbe7051078/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.768210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4068941ed944431b1c2f8c129f56e73, user_id=3cf0d099461a433bad6719cbe7051078, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:04.770945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:04.783113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 893/1758] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:04 2022] PUT /identity/v3/projects/d4068941ed944431b1c2f8c129f56e73/users/3cf0d099461a433bad6719cbe7051078/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:04.793672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:04.794233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:04.794233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:04.825590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] MFA Rules not processed for user `3cf0d099461a433bad6719cbe7051078`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:04.826968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:04.939183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 866/1759] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:04.953780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:05.021833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711085a-2b3e-430e-a813-9a2c33558c78 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:05.027112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:05.027831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:05.028344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:05.028696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:05.042194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:05.043651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:05.054302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 894/1760] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:05.110021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:05.120349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa42fa4-5a25-46ff-829c-2e8b32af480a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:05.125112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aed64b-e7ec-4c6c-ad2b-e96281109b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:05.126069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aed64b-e7ec-4c6c-ad2b-e96281109b1e service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:05.126476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aed64b-e7ec-4c6c-ad2b-e96281109b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:05.126949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aed64b-e7ec-4c6c-ad2b-e96281109b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:05.203123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aed64b-e7ec-4c6c-ad2b-e96281109b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:05.204694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aed64b-e7ec-4c6c-ad2b-e96281109b1e service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:05.216079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 867/1761] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:05.996494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:06.054406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:06.088756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b67de-632b-4669-92bb-e8e6aaff636b service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:06.094180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:06.095292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:06.095580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:06.095861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] PATH_INFO: `/v3/users/e48bbad6334e4e32a652297a6196ebe8` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:06.097085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48bbad6334e4e32a652297a6196ebe8)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:06.099197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:06.199704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48bbad6334e4e32a652297a6196ebe8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.216942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48bbad6334e4e32a652297a6196ebe8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.237827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48bbad6334e4e32a652297a6196ebe8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.238565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48bbad6334e4e32a652297a6196ebe8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.239444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48bbad6334e4e32a652297a6196ebe8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.254283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 895/1762] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:05 2022] DELETE /identity/v3/users/e48bbad6334e4e32a652297a6196ebe8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:06.259721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:06.260226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:06.260337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:06.279447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:06.282730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:06.300696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:06.350281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 868/1763] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:06.360077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:06.361538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:06.449925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397e00-1744-4167-9058-8f3122c02d10 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:06.453887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:06.454650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:06.455013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:06.455362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:06.465300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:06.466999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:06.477066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 896/1764] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:06.815857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:06.817475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:06.839937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29a2f7-e563-47c4-96cd-4a1f6875c241 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:06.859440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:06.860233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:06.860523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:06.860835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] PATH_INFO: `/v3/projects/e491c88c32184fcc9dc795069bcc5a64` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:06.862057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e491c88c32184fcc9dc795069bcc5a64)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:06.864206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:06.967988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e491c88c32184fcc9dc795069bcc5a64', 'request_id': 'req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.977884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e491c88c32184fcc9dc795069bcc5a64', 'request_id': 'req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:06.984854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e491c88c32184fcc9dc795069bcc5a64', 'request_id': 'req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.015540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] The token cache is being invalidate because project e491c88c32184fcc9dc795069bcc5a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:07.015847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.016762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.029231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 869/1765] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:06 2022] DELETE /identity/v3/projects/e491c88c32184fcc9dc795069bcc5a64 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:07.033173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:07.050649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3db52c-907c-4d2c-9a25-6b4f48e8e4a4 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:07.155003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:07.156911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:07.157404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:07.157909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] PATH_INFO: `/v3/users/b719aae0e4ae48598eef4b978f134fb0` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:07.159554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b719aae0e4ae48598eef4b978f134fb0)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:07.161883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:07.175583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:07.266072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b719aae0e4ae48598eef4b978f134fb0'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.287125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b719aae0e4ae48598eef4b978f134fb0'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.324315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b719aae0e4ae48598eef4b978f134fb0'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.325133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b719aae0e4ae48598eef4b978f134fb0'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.325747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8df38af5-d814-40c0-98a9-a7d8c26a4dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b719aae0e4ae48598eef4b978f134fb0'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:07.348564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 897/1766] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:07 2022] DELETE /identity/v3/users/b719aae0e4ae48598eef4b978f134fb0 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:07.947690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:07.949175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:07.983735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e20c9-f2b7-4dcb-8199-6eb1211ddb4c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:08.104156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:08.105300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.105762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.106208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] PATH_INFO: `/v3/projects/1fdc64189cd446569b71a9294a55f7eb` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.107645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fdc64189cd446569b71a9294a55f7eb)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:08.109752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:08.186114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:08.191291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fdc64189cd446569b71a9294a55f7eb', 'request_id': 'req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:08.207131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fdc64189cd446569b71a9294a55f7eb', 'request_id': 'req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:08.213430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fdc64189cd446569b71a9294a55f7eb', 'request_id': 'req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:08.253223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] The token cache is being invalidate because project 1fdc64189cd446569b71a9294a55f7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:08.253572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:08.255482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e76d1c49-f78a-4efa-8bb2-76803b7c84e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:08.264015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 870/1767] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:07 2022] DELETE /identity/v3/projects/1fdc64189cd446569b71a9294a55f7eb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:08.269308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.269796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.269870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.306906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:08.308304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:08.420740 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 898/1768] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:08.428501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.429940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.430308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.472432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:08.474160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:08.498659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 871/1769] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:08.502520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:08.515542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16f624-5970-40f2-b86d-7f2498c6a4c6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:08.522150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:08.523142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.523522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.524076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.525137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:08.525511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:08.527371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:08.540284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 899/1770] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:08.544548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:08.555401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8e2d5-6e45-4cda-9860-c8e6b610cbc1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:08.563119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:08.563793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.564374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.564374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.565079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:08.571963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:08.619533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 872/1771] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:08 2022] POST /identity/v3/projects => generated 401 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:08.620996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:08.632263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba5550-f997-41a4-a854-c50fd8e5ebbf admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:08.778787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:08.779759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.780248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.780790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.782757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:08.800164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:08.850351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 900/1772] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:08 2022] POST /identity/v3/users => generated 301 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:08.854374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:08.868212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507be1a-6f36-486d-b2bf-da294c331926 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:08.874669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:08.875548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.875824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.876107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.876855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:08.878651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:08.894358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 873/1773] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:08.898024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:08.907611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:08.916058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6eb16-6fa2-486b-861f-bdc782931ca8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:08.941905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c671eff8-ce7b-42b7-8ce5-9494f7611a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:08.943245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671eff8-ce7b-42b7-8ce5-9494f7611a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:08.944863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671eff8-ce7b-42b7-8ce5-9494f7611a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:08.945146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c671eff8-ce7b-42b7-8ce5-9494f7611a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:08.945963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671eff8-ce7b-42b7-8ce5-9494f7611a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:08.951984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c671eff8-ce7b-42b7-8ce5-9494f7611a1e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:08.989792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:08.996034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 901/1774] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:09.015044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75467072-fbd3-41df-9f0a-60d14f070b04 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.043406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.044413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.044711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.045003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] PATH_INFO: `/v3/projects/6611582444dd42268dd8ca78170bafce/users/69e4c3c20e4e469daf287686157eab22/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.080287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6611582444dd42268dd8ca78170bafce, user_id=69e4c3c20e4e469daf287686157eab22, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.083022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.104644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 874/1775] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:08 2022] PUT /identity/v3/projects/6611582444dd42268dd8ca78170bafce/users/69e4c3c20e4e469daf287686157eab22/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:09.119310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.119662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.119944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.164519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] MFA Rules not processed for user `69e4c3c20e4e469daf287686157eab22`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:09.165873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:09.282801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.284986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:09.314499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 902/1776] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:09.320303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.405822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd393f4-db21-4923-bd7d-11e452f22bcd None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.422489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.423121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e3ac3-372e-4d58-987f-3214e3cb59f8 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.423248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.423495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.423793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.424505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.430574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.438795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.439712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.440119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.440411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] PATH_INFO: `/v3/users/d86691d8d4684eeabe20fe0213a98392` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.444760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86691d8d4684eeabe20fe0213a98392)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.449179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.491030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 903/1777] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:09 2022] POST /identity/v3/projects => generated 399 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:09.492606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.552095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86691d8d4684eeabe20fe0213a98392'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:09.565256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86691d8d4684eeabe20fe0213a98392'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:09.578325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd794-485f-4b32-a9f9-4490f03cd1cf admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.582064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:09.584923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86691d8d4684eeabe20fe0213a98392'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:09.585736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.586545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.587026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.587204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.588053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.590143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.591906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86691d8d4684eeabe20fe0213a98392'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:09.592754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-374edac9-7b96-4d03-a70c-1fc72531a44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86691d8d4684eeabe20fe0213a98392'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:09.621037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 875/1778] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:09 2022] DELETE /identity/v3/users/d86691d8d4684eeabe20fe0213a98392 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:09.621352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:09.623437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.624044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.624582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.630785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 904/1779] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:09 2022] POST /identity/v3/users => generated 306 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:09.634020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.639813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:09.647613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:09.649345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:09.656452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5782-3c94-4b51-8370-d3bee55eada3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.744641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 876/1780] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:09.748823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.749667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.749956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.750231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.751170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.752798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.756060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.757838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:09.786018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 905/1781] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:09.790774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.806362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6beb8d-d2d2-4e71-950d-29d803ef605e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.813295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.814340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.814752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.815130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] PATH_INFO: `/v3/projects/56604196a1904fbca15a3c0ae0398d27/users/6933cfb2a5be4d16b35691ca08036e5d/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.837445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd943fc8-40b9-43f9-9f10-16a1c65e78f1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:09.841186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=56604196a1904fbca15a3c0ae0398d27, user_id=6933cfb2a5be4d16b35691ca08036e5d, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.841324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:09.841896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:09.842192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:09.842461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:09.846863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.855911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:09.857728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:09.870995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 877/1782] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:05:09.875248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 906/1783] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:09 2022] PUT /identity/v3/projects/56604196a1904fbca15a3c0ae0398d27/users/6933cfb2a5be4d16b35691ca08036e5d/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:09.880292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:09.902030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae966c-2e71-4278-9a50-ef60844398c0 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:10.004236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4774de50-d5dc-46dc-b5d2-efc58d9da465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:10.012974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774de50-d5dc-46dc-b5d2-efc58d9da465 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.013279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774de50-d5dc-46dc-b5d2-efc58d9da465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.013847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4774de50-d5dc-46dc-b5d2-efc58d9da465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.014639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774de50-d5dc-46dc-b5d2-efc58d9da465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:10.016322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4774de50-d5dc-46dc-b5d2-efc58d9da465 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:10.030495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 878/1784] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:10.034602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:10.050899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87f772-e652-4be8-8929-49f51ec834ea admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:10.057108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:10.058352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.058843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.059298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] PATH_INFO: `/v3/projects/56604196a1904fbca15a3c0ae0398d27/users/6933cfb2a5be4d16b35691ca08036e5d/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.061860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56604196a1904fbca15a3c0ae0398d27, user_id=6933cfb2a5be4d16b35691ca08036e5d, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:10.077300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:10.109733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 907/1785] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:10 2022] PUT /identity/v3/projects/56604196a1904fbca15a3c0ae0398d27/users/6933cfb2a5be4d16b35691ca08036e5d/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:10.122430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.122719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.122998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.166561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] MFA Rules not processed for user `6933cfb2a5be4d16b35691ca08036e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:10.168109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:10.188104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:10.334604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 879/1786] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:10.348257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:10.350149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:10.374706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:10.376824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:10.445226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa745d-a85e-4538-b499-2a6d6490db39 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:10.449426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:10.450086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.450368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.450706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.460389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:10.461821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:10.468251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c362f-71e9-4c4f-b08c-2025d8069932 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:10.472018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 908/1787] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:10.473016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:10.474422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.474745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.475026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] PATH_INFO: `/v3/projects/5f5c8c694e154b79af603fb6a8e620b5` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.476261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f5c8c694e154b79af603fb6a8e620b5)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:10.478180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:10.527347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f5c8c694e154b79af603fb6a8e620b5', 'request_id': 'req-3433e98a-fb1c-40b9-96a8-9958300f2b57'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.534190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f5c8c694e154b79af603fb6a8e620b5', 'request_id': 'req-3433e98a-fb1c-40b9-96a8-9958300f2b57'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.545832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f5c8c694e154b79af603fb6a8e620b5', 'request_id': 'req-3433e98a-fb1c-40b9-96a8-9958300f2b57'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.548184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:10.559341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488d385-c93a-4ba6-bf5c-9074a372f15d service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:10.561932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] The token cache is being invalidate because project 5f5c8c694e154b79af603fb6a8e620b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:10.562600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.563302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.565267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 880/1788] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:10 2022] DELETE /identity/v3/projects/5f5c8c694e154b79af603fb6a8e620b5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:10.569030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:10.570194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:10.620564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:10.624249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:10.624912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.625201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.625558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.627806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:10.642567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:10.645917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433e98a-fb1c-40b9-96a8-9958300f2b57 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:10.651721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:10.652888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:10.653068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:10.653345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] PATH_INFO: `/v3/users/3cf0d099461a433bad6719cbe7051078` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:10.654575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf0d099461a433bad6719cbe7051078)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:10.656595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:10.706394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:10.708363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:10.708805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf0d099461a433bad6719cbe7051078'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.717759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf0d099461a433bad6719cbe7051078'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.718370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 909/1789] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:10.730905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf0d099461a433bad6719cbe7051078'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.731609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf0d099461a433bad6719cbe7051078'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.732234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a1215b25-49fc-4e2b-a06b-c3845c3ec6e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf0d099461a433bad6719cbe7051078'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:10.745204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 881/1790] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:10 2022] DELETE /identity/v3/users/3cf0d099461a433bad6719cbe7051078 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:11.042478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.055303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebd06d-d679-494d-908c-8278946601ec service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.121104 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.121965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.122247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.122525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] PATH_INFO: `/v3/projects/d4068941ed944431b1c2f8c129f56e73` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.123736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4068941ed944431b1c2f8c129f56e73)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.125666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.171490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4068941ed944431b1c2f8c129f56e73', 'request_id': 'req-5a83de1a-27ef-4f84-87ff-1d14800259d5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:11.177641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4068941ed944431b1c2f8c129f56e73', 'request_id': 'req-5a83de1a-27ef-4f84-87ff-1d14800259d5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:11.184567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4068941ed944431b1c2f8c129f56e73', 'request_id': 'req-5a83de1a-27ef-4f84-87ff-1d14800259d5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:11.199548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] The token cache is being invalidate because project d4068941ed944431b1c2f8c129f56e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:11.199843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:11.200476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5a83de1a-27ef-4f84-87ff-1d14800259d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:11.202059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 910/1791] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:11 2022] DELETE /identity/v3/projects/d4068941ed944431b1c2f8c129f56e73 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:11.207168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.207670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.208276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.226842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:11.228039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:11.293049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 882/1792] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:11.301696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.302019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.302387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.321285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:11.322496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:11.330407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 911/1793] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:11.335297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.344978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8e8a8-ad99-4b29-a7e1-ba68035d78cb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.349586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.350238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.350528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.350844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.351576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:11.351884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.353495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.364880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 883/1794] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:11.368834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.378084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59617b-62f7-46dc-81ec-68df1bd70f8e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.383414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.384075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.384359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.384641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.385336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.387388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.421113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 912/1795] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:11.421911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.432015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba53e49-9789-480d-a5a6-76ed5b3e0a76 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.508515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:11.529538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.530610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.531009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.531679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.532463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.535754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.571172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 884/1796] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:11.573911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.592511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27240a2-bb2f-46cd-9d70-079844e945a5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.603421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.604293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.604591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.605007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.605806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.607493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.632591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 913/1797] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:11.636893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.657719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf022c-0a22-4f7f-a7b8-b10b461a599c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.662483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5e192-c14e-4fed-9e3b-eb589d2e6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.663826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e192-c14e-4fed-9e3b-eb589d2e6cff admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.664113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e192-c14e-4fed-9e3b-eb589d2e6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.664392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5e192-c14e-4fed-9e3b-eb589d2e6cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.665163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5e192-c14e-4fed-9e3b-eb589d2e6cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.666850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5e192-c14e-4fed-9e3b-eb589d2e6cff admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.677876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 885/1798] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:11.681990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.694014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb4d97-af18-42d0-9004-8709120312d6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.695147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:11.701166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.710019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.710474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.711183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] PATH_INFO: `/v3/projects/5bee004acc5f48b1a3dc63ad25c6c80c/users/a5903f0eb9bc4d13a1c0e02b35ef8772/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.726876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bee004acc5f48b1a3dc63ad25c6c80c, user_id=a5903f0eb9bc4d13a1c0e02b35ef8772, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.729547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.743640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 914/1799] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:11 2022] PUT /identity/v3/projects/5bee004acc5f48b1a3dc63ad25c6c80c/users/a5903f0eb9bc4d13a1c0e02b35ef8772/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:11.753838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.754127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.754409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.782321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] MFA Rules not processed for user `a5903f0eb9bc4d13a1c0e02b35ef8772`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:11.783614 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:11.862301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 886/1800] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:11.867068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.929407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dfd28-2e2f-440c-a96a-76b33bdc65aa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:11.934507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:11.935235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:11.935518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:11.935787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:11.936483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:11.938371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:11.965129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 915/1801] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:11.968890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:11.979761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ae6b4-20fc-4c7d-9621-0febf6b21c36 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.044422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.045105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.045390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.045664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.046359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.048309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.070748 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 887/1802] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:11 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:12.075238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.088131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf178d1-a90e-413a-b4e3-247e46b711eb admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.093345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.094071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.094359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.094660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.095618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.097344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.117322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 916/1803] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:12.121331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.131854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8e05b-27c0-460c-b615-fe6c1f2b4769 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.136801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.137731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.138018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.138299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] PATH_INFO: `/v3/projects/252abae63ca9476eb7d4ce0dc19f8455/users/c2187d50453a4c0191586c507d4b8ce5/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.155490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=252abae63ca9476eb7d4ce0dc19f8455, user_id=c2187d50453a4c0191586c507d4b8ce5, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.158119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.177615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 888/1804] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:12 2022] PUT /identity/v3/projects/252abae63ca9476eb7d4ce0dc19f8455/users/c2187d50453a4c0191586c507d4b8ce5/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:12.181555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.192878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a47c44-af7f-4bd0-83e6-1c0b4a760896 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.265960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20acc62-a9d0-4ac0-9e70-919e22b55f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.266776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20acc62-a9d0-4ac0-9e70-919e22b55f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.267007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20acc62-a9d0-4ac0-9e70-919e22b55f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.267315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20acc62-a9d0-4ac0-9e70-919e22b55f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.268032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20acc62-a9d0-4ac0-9e70-919e22b55f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.269698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20acc62-a9d0-4ac0-9e70-919e22b55f9c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.287348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 917/1805] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:12.291580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.303647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebe48e-9443-4b86-9b1a-1b6096428c7f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.308848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.309900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.310179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.310463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] PATH_INFO: `/v3/projects/252abae63ca9476eb7d4ce0dc19f8455/users/c2187d50453a4c0191586c507d4b8ce5/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.312757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] RBAC: Authorizing `identity:create_grant(project_id=252abae63ca9476eb7d4ce0dc19f8455, user_id=c2187d50453a4c0191586c507d4b8ce5, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.315496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.326465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 889/1806] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:12 2022] PUT /identity/v3/projects/252abae63ca9476eb7d4ce0dc19f8455/users/c2187d50453a4c0191586c507d4b8ce5/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:12.337493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.337818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.338194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.367926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] MFA Rules not processed for user `c2187d50453a4c0191586c507d4b8ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:12.369661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:12.413663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:12.453983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 918/1807] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:12.466048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.471728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:12.485957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:12.557422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702826b-aeb9-4e15-8e83-a78f7374379c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.561016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a08f3-774a-4f4a-af70-11efe32c8af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.561657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a08f3-774a-4f4a-af70-11efe32c8af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.561940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a08f3-774a-4f4a-af70-11efe32c8af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.562215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a08f3-774a-4f4a-af70-11efe32c8af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.571984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a08f3-774a-4f4a-af70-11efe32c8af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.573471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a08f3-774a-4f4a-af70-11efe32c8af1 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.583421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 890/1808] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:12.677426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.777863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aef622-236c-49c8-afc6-d49a174dc3dd None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:12.782915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:12.784091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.784381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.784660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] PATH_INFO: `/v3/users/a5903f0eb9bc4d13a1c0e02b35ef8772` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.786139 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5903f0eb9bc4d13a1c0e02b35ef8772)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:12.788093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:12.845391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5903f0eb9bc4d13a1c0e02b35ef8772'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:12.853857 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5903f0eb9bc4d13a1c0e02b35ef8772'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:12.869661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5903f0eb9bc4d13a1c0e02b35ef8772'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:12.870364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5903f0eb9bc4d13a1c0e02b35ef8772'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:12.871012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5903f0eb9bc4d13a1c0e02b35ef8772'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:12.884075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 919/1809] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:12 2022] DELETE /identity/v3/users/a5903f0eb9bc4d13a1c0e02b35ef8772 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:12.888677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:12.888994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:12.889275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:12.908205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:12.909489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:12.969954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 891/1810] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:12.979782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:12.981361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:13.057085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5339e2d-0765-4d9b-a356-e6633005f445 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:13.060843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:13.061654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:13.062009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:13.062353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:13.080029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:13.081573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:13.086663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:13.096537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 920/1811] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:13.395529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:13.397630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:13.410336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b291e1e-c989-40d7-b27f-a3cc8ba467aa None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:13.415065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:13.415946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:13.416224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:13.416499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] PATH_INFO: `/v3/projects/5bee004acc5f48b1a3dc63ad25c6c80c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:13.417684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bee004acc5f48b1a3dc63ad25c6c80c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:13.419677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:13.468656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bee004acc5f48b1a3dc63ad25c6c80c', 'request_id': 'req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.475266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bee004acc5f48b1a3dc63ad25c6c80c', 'request_id': 'req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.481296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bee004acc5f48b1a3dc63ad25c6c80c', 'request_id': 'req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.513914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] The token cache is being invalidate because project 5bee004acc5f48b1a3dc63ad25c6c80c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:13.514211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.523120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.525735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 892/1812] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:13 2022] DELETE /identity/v3/projects/5bee004acc5f48b1a3dc63ad25c6c80c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:13.529593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:13.546648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdd4b-41c4-4c7e-b93e-7aa1826a2fca service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:13.635423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:13.637141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:13.638086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:13.638590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] PATH_INFO: `/v3/users/c2187d50453a4c0191586c507d4b8ce5` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:13.640275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2187d50453a4c0191586c507d4b8ce5)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:13.642659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:13.746078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2187d50453a4c0191586c507d4b8ce5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.787374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2187d50453a4c0191586c507d4b8ce5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.811258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2187d50453a4c0191586c507d4b8ce5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.812377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2187d50453a4c0191586c507d4b8ce5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.813489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a92d40fd-2eea-4762-96f9-450fd06cd3bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2187d50453a4c0191586c507d4b8ce5'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:13.833823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 921/1813] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:13 2022] DELETE /identity/v3/users/c2187d50453a4c0191586c507d4b8ce5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:14.290505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.292167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:14.306610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2d7f2-7c7b-4cef-a89a-50da69437f18 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.345812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:14.371224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.372101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.372386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.372668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] PATH_INFO: `/v3/projects/252abae63ca9476eb7d4ce0dc19f8455` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.373861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252abae63ca9476eb7d4ce0dc19f8455)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.375830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.427502 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252abae63ca9476eb7d4ce0dc19f8455', 'request_id': 'req-02d125c5-86da-495a-a75f-d342edde68f9'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:14.434224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252abae63ca9476eb7d4ce0dc19f8455', 'request_id': 'req-02d125c5-86da-495a-a75f-d342edde68f9'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:14.440451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252abae63ca9476eb7d4ce0dc19f8455', 'request_id': 'req-02d125c5-86da-495a-a75f-d342edde68f9'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:14.453758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] The token cache is being invalidate because project 252abae63ca9476eb7d4ce0dc19f8455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:14.454143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:14.454890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-02d125c5-86da-495a-a75f-d342edde68f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:14.456551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 893/1814] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:14 2022] DELETE /identity/v3/projects/252abae63ca9476eb7d4ce0dc19f8455 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:14.462346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.462667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.462957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.480368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:14.481581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:14.538532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 922/1815] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:14.546462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.546792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.547075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.565686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:14.567416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:14.575397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 894/1816] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:14.579632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.589884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b9fd8-ed79-4720-a177-e90f2d6f74e9 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.594411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.595103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.595390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.595668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.596401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:14.596701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.598318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.609549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 923/1817] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:05:14.613687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.623270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054aabb-a9b1-4ad8-bb28-58fc2cb9d489 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.627800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.628453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.628739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.629015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.630333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.632261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.657087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 895/1818] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:14 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:14.660914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.667158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:14.671162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0d35-1363-4f65-8f5e-3b270ceb6417 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.727711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.728309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.728598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.728879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.729579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.731524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.751033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 924/1819] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:14 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:14.755721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.766032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f5261-ea67-45f9-8fde-d5757732c378 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.770683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.771347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.771636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.771916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.772646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.774238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.786548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 896/1820] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:14.789122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.799196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab35e98-b537-4930-b7c1-7b76ac4ee298 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.804366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c0ef-9b05-43ff-8563-e12199919e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.805022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c0ef-9b05-43ff-8563-e12199919e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.805309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c0ef-9b05-43ff-8563-e12199919e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.805584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c0ef-9b05-43ff-8563-e12199919e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.806305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c0ef-9b05-43ff-8563-e12199919e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.807934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02c0ef-9b05-43ff-8563-e12199919e6e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.819571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 925/1821] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:14.823836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.833947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb39f2d-5545-45b7-bc1f-18acc0e3672a admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:14.839160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:14.840194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.840483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.840761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] PATH_INFO: `/v3/projects/9652d3c9314f4b4cad9ac7ae8eff2e16/users/b7e6b7e10aad48ff8f082d2cbe5044fe/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.855646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9652d3c9314f4b4cad9ac7ae8eff2e16, user_id=b7e6b7e10aad48ff8f082d2cbe5044fe, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:14.858314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:14.872250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 897/1822] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:14 2022] PUT /identity/v3/projects/9652d3c9314f4b4cad9ac7ae8eff2e16/users/b7e6b7e10aad48ff8f082d2cbe5044fe/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:14.883157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:14.883455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:14.883739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:14.911236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] MFA Rules not processed for user `b7e6b7e10aad48ff8f082d2cbe5044fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:14.912488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:14.975872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 926/1823] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:14.986637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:14.988346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:15.051106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaad098-9a29-4d72-bef9-4fa82eb3b951 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:15.059232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765118f-0747-4a4b-b0be-5e4334998b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:15.060065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765118f-0747-4a4b-b0be-5e4334998b52 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:15.060854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765118f-0747-4a4b-b0be-5e4334998b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:15.060854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765118f-0747-4a4b-b0be-5e4334998b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:15.071437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765118f-0747-4a4b-b0be-5e4334998b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:15.073487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765118f-0747-4a4b-b0be-5e4334998b52 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:15.082751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 898/1824] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:15.908297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:15.909811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:15.967924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ab2ba-144f-4ff2-993c-0056610fb505 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:15.971695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:15.972473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:15.972867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:15.973247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:15.975347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:15.985507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:15.986969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131e8b1-13d1-45ff-a9fe-f8602958d14b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:15.994982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 927/1825] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:17.228042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9a9cf-4dcf-42a2-86c7-da1ec9685161 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:17.229442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9a9cf-4dcf-42a2-86c7-da1ec9685161 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:17.229630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9a9cf-4dcf-42a2-86c7-da1ec9685161 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:17.232637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 899/1826] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:05:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:05:17.298808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4166-bd57-44c8-9d58-9bef39ea4166 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:17.299713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4166-bd57-44c8-9d58-9bef39ea4166 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:17.300201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4166-bd57-44c8-9d58-9bef39ea4166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:17.338566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-49fb4166-bd57-44c8-9d58-9bef39ea4166 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:17.341042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-49fb4166-bd57-44c8-9d58-9bef39ea4166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:17.362900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:17.491548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 928/1827] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:17.520761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9a9cf-4dcf-42a2-86c7-da1ec9685161 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:17.523190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-46a9a9cf-4dcf-42a2-86c7-da1ec9685161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:17.554599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:17.609037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9a9cf-4dcf-42a2-86c7-da1ec9685161 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:17.624179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e45fd6-c775-4b15-bfb0-39616aaa25cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:17.625816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e45fd6-c775-4b15-bfb0-39616aaa25cd service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:17.629364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e45fd6-c775-4b15-bfb0-39616aaa25cd service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:17.630250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e45fd6-c775-4b15-bfb0-39616aaa25cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:17.657242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e45fd6-c775-4b15-bfb0-39616aaa25cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:17.658883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e45fd6-c775-4b15-bfb0-39616aaa25cd service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:17.681294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 900/1828] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:18.630528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69468-5938-49db-b087-a6dbf4bdd556 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:18.631386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69468-5938-49db-b087-a6dbf4bdd556 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:18.631840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69468-5938-49db-b087-a6dbf4bdd556 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:18.633225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 929/1829] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:05:18.646469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485fe6-b2e7-4fc7-a499-b4638b042d35 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:18.647134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485fe6-b2e7-4fc7-a499-b4638b042d35 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:18.647134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df485fe6-b2e7-4fc7-a499-b4638b042d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:18.689456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-df485fe6-b2e7-4fc7-a499-b4638b042d35 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:18.690974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-df485fe6-b2e7-4fc7-a499-b4638b042d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:18.710794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 901/1830] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:18.733869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69468-5938-49db-b087-a6dbf4bdd556 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:18.735675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-64d69468-5938-49db-b087-a6dbf4bdd556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:18.764569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69468-5938-49db-b087-a6dbf4bdd556 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:18.771029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56c0d3-4e92-4244-869b-994a15c8b4c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:18.771965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56c0d3-4e92-4244-869b-994a15c8b4c1 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:18.776256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56c0d3-4e92-4244-869b-994a15c8b4c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:18.776687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56c0d3-4e92-4244-869b-994a15c8b4c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:18.797242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56c0d3-4e92-4244-869b-994a15c8b4c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:18.799491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56c0d3-4e92-4244-869b-994a15c8b4c1 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:18.821138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 930/1831] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:28.672394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=7 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:28.807988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=30 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:35.139441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2ff02-fb29-47a4-b23e-8f30ed7d1a43 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:35.139940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2ff02-fb29-47a4-b23e-8f30ed7d1a43 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:35.140045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2ff02-fb29-47a4-b23e-8f30ed7d1a43 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:35.141663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 902/1832] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:05:35.150995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:35.151448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:35.151647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:35.171022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:35.172237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:35.181829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 931/1833] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:35.193979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2ff02-fb29-47a4-b23e-8f30ed7d1a43 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:35.208760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2ff02-fb29-47a4-b23e-8f30ed7d1a43 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:35.214100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195a771-84ef-4ead-91c1-5122f64d2442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:35.214777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195a771-84ef-4ead-91c1-5122f64d2442 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:35.215060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195a771-84ef-4ead-91c1-5122f64d2442 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:35.215332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195a771-84ef-4ead-91c1-5122f64d2442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:35.225825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195a771-84ef-4ead-91c1-5122f64d2442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:35.227439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195a771-84ef-4ead-91c1-5122f64d2442 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:35.237969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 903/1834] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:35.462123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:35.463727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:35.540817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674189c-7a2a-4d4d-8022-a19dfff3b01d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:35.545416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:35.546296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:35.546581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:35.546912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] PATH_INFO: `/v3/users/69e4c3c20e4e469daf287686157eab22` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:35.548115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e4c3c20e4e469daf287686157eab22)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:35.550037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:35.597214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e4c3c20e4e469daf287686157eab22'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:35.604162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e4c3c20e4e469daf287686157eab22'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:35.616289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e4c3c20e4e469daf287686157eab22'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:35.616930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e4c3c20e4e469daf287686157eab22'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:35.617530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e4c3c20e4e469daf287686157eab22'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:35.630385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 932/1835] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:35 2022] DELETE /identity/v3/users/69e4c3c20e4e469daf287686157eab22 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:35.634869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:35.635163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:35.635446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:35.652648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:35.653843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:35.710949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 904/1836] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:35.720290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:35.721684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:35.778195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb7c40-89b4-4dff-afdd-990e9f4e3471 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:35.781711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:35.782356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:35.782667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:35.782952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:35.792116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:35.793536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:35.803641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 933/1837] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:36.086055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:36.087788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:36.104383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf2e89-0344-4dfa-9b5d-babf5a139771 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:36.109511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:36.110450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:36.110768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:36.111049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] PATH_INFO: `/v3/projects/6611582444dd42268dd8ca78170bafce` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:36.112805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6611582444dd42268dd8ca78170bafce)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:36.115777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:36.164633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6611582444dd42268dd8ca78170bafce', 'request_id': 'req-19cc40d8-4e57-453d-b557-a1c2cbd1a829'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.171116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6611582444dd42268dd8ca78170bafce', 'request_id': 'req-19cc40d8-4e57-453d-b557-a1c2cbd1a829'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.177457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6611582444dd42268dd8ca78170bafce', 'request_id': 'req-19cc40d8-4e57-453d-b557-a1c2cbd1a829'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.190519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] The token cache is being invalidate because project 6611582444dd42268dd8ca78170bafce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:36.190836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.191512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.193107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 905/1838] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:36 2022] DELETE /identity/v3/projects/6611582444dd42268dd8ca78170bafce => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:36.197057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:36.206840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c0ff7-cc2e-4731-acf2-5720c76cb41d service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:36.218623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:36.270923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:36.271408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:36.271696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:36.271976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] PATH_INFO: `/v3/users/6933cfb2a5be4d16b35691ca08036e5d` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:36.273245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6933cfb2a5be4d16b35691ca08036e5d)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:36.275514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:36.321592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6933cfb2a5be4d16b35691ca08036e5d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.328639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6933cfb2a5be4d16b35691ca08036e5d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.340383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6933cfb2a5be4d16b35691ca08036e5d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.341031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6933cfb2a5be4d16b35691ca08036e5d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.341651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-73a7c5a7-46c3-4a73-a0c9-9e157884586a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6933cfb2a5be4d16b35691ca08036e5d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.354125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 934/1839] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:36 2022] DELETE /identity/v3/users/6933cfb2a5be4d16b35691ca08036e5d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:36.687424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:36.688967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:36.703778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc40d8-4e57-453d-b557-a1c2cbd1a829 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:36.768431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:36.769857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:36.770141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:36.770414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] PATH_INFO: `/v3/projects/56604196a1904fbca15a3c0ae0398d27` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:36.771628 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56604196a1904fbca15a3c0ae0398d27)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:36.773538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:36.785136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:36.820914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56604196a1904fbca15a3c0ae0398d27', 'request_id': 'req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.826846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56604196a1904fbca15a3c0ae0398d27', 'request_id': 'req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.832895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56604196a1904fbca15a3c0ae0398d27', 'request_id': 'req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.845530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] The token cache is being invalidate because project 56604196a1904fbca15a3c0ae0398d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:36.845826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.846466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64ba3dde-8cc3-4be2-b0df-a6b2f63b335e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:36.848096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 906/1840] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:36 2022] DELETE /identity/v3/projects/56604196a1904fbca15a3c0ae0398d27 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:36.853001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:36.853242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:36.853525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:36.871276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:36.872487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:36.930684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 935/1841] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:36.939012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:36.939300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:36.939580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:36.958508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:36.959716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:36.967451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 907/1842] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:36.971409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:36.981196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60578192-f381-4790-8c8e-cad0dcbbcf69 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:36.985746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:36.986401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:36.986745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:36.987046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:36.987805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:36.988140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:36.989746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.006341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 936/1843] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:37.008075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.017263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e661bd-68d5-4e84-b60a-b9afa1deef74 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.022585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.023271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.023558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.024096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.024800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.026754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.052309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 908/1844] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:37 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:37.056477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.067585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c613c-2f7d-4828-b7ee-7a6a806a76b8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.089297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:37.137225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.138136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.138429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.138745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.139490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.141445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.162469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 937/1845] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:37 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:37.166776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.177524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8602030-56e6-4502-bd9d-1ea8300f3430 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.182124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.182815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.183103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.183386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.184152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.185767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.197147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 909/1846] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:37.200986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.210695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430125d8-89f4-472c-9885-1ca35a385c72 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.215166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088e0a-9c9c-4e9e-9bf0-ec3b4579fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.215813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088e0a-9c9c-4e9e-9bf0-ec3b4579fa5f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.216095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088e0a-9c9c-4e9e-9bf0-ec3b4579fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.216369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088e0a-9c9c-4e9e-9bf0-ec3b4579fa5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.217111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088e0a-9c9c-4e9e-9bf0-ec3b4579fa5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.218686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088e0a-9c9c-4e9e-9bf0-ec3b4579fa5f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.228915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 938/1847] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:37.232591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.242993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e4db-0574-457c-bda9-08983f08378d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.247615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.248489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.248776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.249060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] PATH_INFO: `/v3/projects/0d0248df91a64957b79f4fbcdc4a4290/users/bc6083f0d07e41f5a7753795c95359a1/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.265120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0248df91a64957b79f4fbcdc4a4290, user_id=bc6083f0d07e41f5a7753795c95359a1, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.267830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.283277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 910/1848] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:37 2022] PUT /identity/v3/projects/0d0248df91a64957b79f4fbcdc4a4290/users/bc6083f0d07e41f5a7753795c95359a1/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:37.293410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.293702 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.293983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.321315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] MFA Rules not processed for user `bc6083f0d07e41f5a7753795c95359a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:37.322525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:37.386200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 939/1849] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:37.390598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.459227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739b2a5-5f88-4d52-956b-e5a7063717d3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.463933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.464625 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.464912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.465189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.465879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.467808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.493434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 911/1850] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:37 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:37.497414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.508074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1120373-8a16-4d95-b7a3-ba2cfed4f1c5 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.563113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.563786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.564075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.564357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.565073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.567053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.590911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 940/1851] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:37 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:37.592326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.603191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76031b3a-8649-4622-9999-b95373a3634d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.607899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.608566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.608853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.609136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.609875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.611492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.622998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 912/1852] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:37.627141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.638061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e35b02-51ce-4f6f-bd4b-69f44864cef5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.644034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.645199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.645415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.645783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] PATH_INFO: `/v3/projects/3d25e6eba31e4159a938fb450270b6d0/users/fd9c5942dbc44a7fb1c71ab7a3d8a38f/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.665887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d25e6eba31e4159a938fb450270b6d0, user_id=fd9c5942dbc44a7fb1c71ab7a3d8a38f, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.668598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.672137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:37.684070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 941/1853] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:37 2022] PUT /identity/v3/projects/3d25e6eba31e4159a938fb450270b6d0/users/fd9c5942dbc44a7fb1c71ab7a3d8a38f/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:37.687978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.697843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745a5fa-7cfe-41ab-97b9-5a50b90e750e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.705553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:37.765240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765d394-d05e-495c-a3c4-329427e66d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.766745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765d394-d05e-495c-a3c4-329427e66d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.766745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765d394-d05e-495c-a3c4-329427e66d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.766745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765d394-d05e-495c-a3c4-329427e66d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.767738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765d394-d05e-495c-a3c4-329427e66d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.769555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765d394-d05e-495c-a3c4-329427e66d9b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.780724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 913/1854] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:37.785489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.796026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636f4b-c3c1-4e36-9495-e4b3709fcae5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:37.804217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:37.805186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.805531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.805875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] PATH_INFO: `/v3/projects/3d25e6eba31e4159a938fb450270b6d0/users/fd9c5942dbc44a7fb1c71ab7a3d8a38f/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.807932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d25e6eba31e4159a938fb450270b6d0, user_id=fd9c5942dbc44a7fb1c71ab7a3d8a38f, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:37.810781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:37.823044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 942/1855] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:37 2022] PUT /identity/v3/projects/3d25e6eba31e4159a938fb450270b6d0/users/fd9c5942dbc44a7fb1c71ab7a3d8a38f/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:37.833096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:37.833386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:37.833667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:37.859922 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] MFA Rules not processed for user `fd9c5942dbc44a7fb1c71ab7a3d8a38f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:37.861152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:37.934930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 914/1856] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:37.945617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:37.947360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:38.003166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c2216-0db9-4075-bef4-c8bec44b163c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:38.006844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:38.007481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:38.007764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:38.008036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:38.017112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:38.018530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:38.028492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 943/1857] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:38.125293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:38.135007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44bba-3e0f-456f-92d9-0a437f82419a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:38.138730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258239f-6d52-403e-b2d4-17946fb86255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:38.139379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258239f-6d52-403e-b2d4-17946fb86255 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:38.139664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258239f-6d52-403e-b2d4-17946fb86255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:38.139942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258239f-6d52-403e-b2d4-17946fb86255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:38.200081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258239f-6d52-403e-b2d4-17946fb86255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:38.201536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258239f-6d52-403e-b2d4-17946fb86255 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:38.211254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 915/1858] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:44.952757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:44.954547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:45.038870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebb4b4-e594-405c-bf29-4b2646e285b0 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:45.044019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:45.045009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:45.045302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:45.045579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] PATH_INFO: `/v3/users/b7e6b7e10aad48ff8f082d2cbe5044fe` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:45.047040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e6b7e10aad48ff8f082d2cbe5044fe)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:45.048993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:45.102422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e6b7e10aad48ff8f082d2cbe5044fe'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.111035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e6b7e10aad48ff8f082d2cbe5044fe'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.125160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e6b7e10aad48ff8f082d2cbe5044fe'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.125869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e6b7e10aad48ff8f082d2cbe5044fe'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.126486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e6b7e10aad48ff8f082d2cbe5044fe'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.138763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 944/1859] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:44 2022] DELETE /identity/v3/users/b7e6b7e10aad48ff8f082d2cbe5044fe => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:45.143264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:45.143880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:45.143880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:45.162238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:45.163519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:45.190091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:45.223254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 916/1860] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:45.233140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:45.234791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:45.242069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:45.295513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473c64d-0a58-4c93-8257-72e22dee874e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:45.299101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6c78-2c78-4268-8294-9b270bdb533b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:45.299751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6c78-2c78-4268-8294-9b270bdb533b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:45.300074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6c78-2c78-4268-8294-9b270bdb533b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:45.300962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6c78-2c78-4268-8294-9b270bdb533b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:45.310193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6c78-2c78-4268-8294-9b270bdb533b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:45.311708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6c78-2c78-4268-8294-9b270bdb533b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:45.323720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 945/1861] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:45.839417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:45.841395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:45.866063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc3c5f-2f24-42df-90f7-3f8a4809655f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:45.875958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:45.877355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:45.877653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:45.877933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] PATH_INFO: `/v3/projects/9652d3c9314f4b4cad9ac7ae8eff2e16` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:45.880653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9652d3c9314f4b4cad9ac7ae8eff2e16)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:45.883028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:45.948274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9652d3c9314f4b4cad9ac7ae8eff2e16', 'request_id': 'req-4c585ef0-d1ac-4466-8502-9b3d6163ac18'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.959271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9652d3c9314f4b4cad9ac7ae8eff2e16', 'request_id': 'req-4c585ef0-d1ac-4466-8502-9b3d6163ac18'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.965702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9652d3c9314f4b4cad9ac7ae8eff2e16', 'request_id': 'req-4c585ef0-d1ac-4466-8502-9b3d6163ac18'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.982569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] The token cache is being invalidate because project 9652d3c9314f4b4cad9ac7ae8eff2e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:45.983094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.984202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4c585ef0-d1ac-4466-8502-9b3d6163ac18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:45.986406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 917/1862] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:45 2022] DELETE /identity/v3/projects/9652d3c9314f4b4cad9ac7ae8eff2e16 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:45.991242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:45.991731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:45.992026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.012216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:46.013919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:46.093633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 946/1863] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:46.102397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.102742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.103027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.121514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:46.122749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:46.131261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 918/1864] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:46.136021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.147064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fbc4a-0fb8-4d6b-9879-ab49146346c8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.152374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.153155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.153556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.153976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.154838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:46.155291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.156926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.170665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 947/1865] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:05:46.174143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.185331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de93680-d681-4931-a517-3a34cb8c49dc None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.190303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.191633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.191917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.192196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.192904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.194915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.236157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 919/1866] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:46 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:46.238492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.254133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cff943-1890-4733-b0c6-dbd18c9dcb19 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.333609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.334522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.335053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.335480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.336485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.338880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.365014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 948/1867] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:46 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:46.369229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.380517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ca5ce-e4f2-427f-9a21-729649292398 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.389143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.389917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.390353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.390643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.391463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.393380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.404635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 920/1868] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:46.408784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.418767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff3c28-345e-4347-b293-edc22e6dd309 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.423308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8177-fd0a-4921-b0af-eedee5cd5fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.423966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8177-fd0a-4921-b0af-eedee5cd5fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.424249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8177-fd0a-4921-b0af-eedee5cd5fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.424522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8177-fd0a-4921-b0af-eedee5cd5fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.425762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8177-fd0a-4921-b0af-eedee5cd5fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.427377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8177-fd0a-4921-b0af-eedee5cd5fa3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.438897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 949/1869] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:46.443664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.454172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e40fc-f991-4af2-a7ff-f57f55400b9d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.458937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.459832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.460121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.460400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] PATH_INFO: `/v3/projects/ebe64309b53349f5acc1e7d352271a75/users/2b8d88d1b386446b91146f8ff59b3e31/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.477420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe64309b53349f5acc1e7d352271a75, user_id=2b8d88d1b386446b91146f8ff59b3e31, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.480127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.494693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 921/1870] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:46 2022] PUT /identity/v3/projects/ebe64309b53349f5acc1e7d352271a75/users/2b8d88d1b386446b91146f8ff59b3e31/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:46.505586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.505853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.506136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.536605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:46.542555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] MFA Rules not processed for user `2b8d88d1b386446b91146f8ff59b3e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:46.544097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:46.639033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 950/1871] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:46.644722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.716658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:46.730730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ecdd6-9a13-4087-ac80-b58f2ecc3956 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.736346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.738093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.738395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.738699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.739405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.741391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.772376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 922/1872] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:46 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:46.773947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.786033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db80b96-fe81-492b-a35b-1f67a97f1086 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.854403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.855302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.855712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.856500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.857303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.859817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.888543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 951/1873] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:46 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:46.892825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.906485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5917198-f988-4720-b587-635e360016a2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.915106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.915867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.916222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.916568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.917437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.919444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.934774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 923/1874] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:46.936646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:46.949297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851e6c9-678c-442e-8641-6c02a15d4158 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:46.954779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:46.955755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:46.957057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:46.957057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] PATH_INFO: `/v3/projects/5e8bd28e1cc644c3967e426164d7a964/users/34a9cfaaa4d047048302cb6dd804ebd4/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:46.976540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bd28e1cc644c3967e426164d7a964, user_id=34a9cfaaa4d047048302cb6dd804ebd4, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:46.979384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:46.994212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 952/1875] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:46 2022] PUT /identity/v3/projects/5e8bd28e1cc644c3967e426164d7a964/users/34a9cfaaa4d047048302cb6dd804ebd4/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:46.998295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:47.008585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bfab66-88f8-4407-802c-26c0c3c60b31 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:47.075415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:47.076140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.076458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.076743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.077491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:47.079157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:47.093874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 924/1876] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:47.098363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:47.108974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee5a75-1553-465b-bfc1-dccd549c2492 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:47.114399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52901b7-1384-40a5-a2a2-c52c8f423184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:47.115365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52901b7-1384-40a5-a2a2-c52c8f423184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.115824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52901b7-1384-40a5-a2a2-c52c8f423184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.116101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52901b7-1384-40a5-a2a2-c52c8f423184 admin admin] PATH_INFO: `/v3/projects/5e8bd28e1cc644c3967e426164d7a964/users/34a9cfaaa4d047048302cb6dd804ebd4/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.118060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52901b7-1384-40a5-a2a2-c52c8f423184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bd28e1cc644c3967e426164d7a964, user_id=34a9cfaaa4d047048302cb6dd804ebd4, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:47.120772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52901b7-1384-40a5-a2a2-c52c8f423184 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:47.123331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:47.125951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.133353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 953/1877] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:47 2022] PUT /identity/v3/projects/5e8bd28e1cc644c3967e426164d7a964/users/34a9cfaaa4d047048302cb6dd804ebd4/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:47.140332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12755d6c-420a-4c65-9aba-5d0842e10b41 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:47.143570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.145382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.145382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.174938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] MFA Rules not processed for user `34a9cfaaa4d047048302cb6dd804ebd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:47.176261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.207655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:47.208938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.209362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.209812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] PATH_INFO: `/v3/users/bc6083f0d07e41f5a7753795c95359a1` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.211249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc6083f0d07e41f5a7753795c95359a1)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:47.214064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:47.223420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:47.259305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 954/1878] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:47.268319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc6083f0d07e41f5a7753795c95359a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:47.275301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:47.277072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.277656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc6083f0d07e41f5a7753795c95359a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:47.286020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:47.292517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc6083f0d07e41f5a7753795c95359a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:47.293602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc6083f0d07e41f5a7753795c95359a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:47.294472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-04b787d9-4615-4a22-a89b-80e20bfb9d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc6083f0d07e41f5a7753795c95359a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:47.306859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 925/1879] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:47 2022] DELETE /identity/v3/users/bc6083f0d07e41f5a7753795c95359a1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:47.311325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.311824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.312246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.329886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:47.331146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.334050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcfdc9-00d2-4fd8-b707-3fa0c8b14e35 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:47.388848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:47.390231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.390665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.391131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.394739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.402598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 926/1880] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:47.415117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:47.416963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.493981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:47.496184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:47.500597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c845b0f-12b0-46db-a111-f226a83c1c26 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:47.505903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:47.506567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.506894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.507176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.507527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 955/1881] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:47.520295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:47.522348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:47.536717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 927/1882] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:47.907504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:47.908956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:47.931224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bed2a-88d4-4011-92f7-5ec66768571b service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:47.937345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:47.938453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:47.938808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:47.939151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] PATH_INFO: `/v3/projects/0d0248df91a64957b79f4fbcdc4a4290` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:47.940442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0248df91a64957b79f4fbcdc4a4290)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:47.942996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:47.997679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:48.001012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0248df91a64957b79f4fbcdc4a4290', 'request_id': 'req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.009162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0248df91a64957b79f4fbcdc4a4290', 'request_id': 'req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.014867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0248df91a64957b79f4fbcdc4a4290', 'request_id': 'req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.028171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] The token cache is being invalidate because project 0d0248df91a64957b79f4fbcdc4a4290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:48.028464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.029220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.030825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 956/1883] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:47 2022] DELETE /identity/v3/projects/0d0248df91a64957b79f4fbcdc4a4290 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:48.035439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:48.045281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5d345-d7ff-4060-af6a-9a6202e3873a service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:48.107469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:48.108420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:48.109209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:48.109209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] PATH_INFO: `/v3/users/fd9c5942dbc44a7fb1c71ab7a3d8a38f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:48.110386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9c5942dbc44a7fb1c71ab7a3d8a38f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:48.113367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:48.146512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:48.168042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9c5942dbc44a7fb1c71ab7a3d8a38f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.176591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9c5942dbc44a7fb1c71ab7a3d8a38f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.192255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9c5942dbc44a7fb1c71ab7a3d8a38f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.192991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9c5942dbc44a7fb1c71ab7a3d8a38f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.193604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0a4d1646-71e0-49e0-bcc9-b3cd03434e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9c5942dbc44a7fb1c71ab7a3d8a38f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.208656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 928/1884] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:48 2022] DELETE /identity/v3/users/fd9c5942dbc44a7fb1c71ab7a3d8a38f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:48.560440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:48.562018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:48.578426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5306a9-52fc-4eb5-8f1a-b6a0da2001da admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:48.647597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:48.648865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:48.649050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:48.649311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] PATH_INFO: `/v3/projects/3d25e6eba31e4159a938fb450270b6d0` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:48.651324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d25e6eba31e4159a938fb450270b6d0)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:48.653609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:48.708803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d25e6eba31e4159a938fb450270b6d0', 'request_id': 'req-0bc7a487-7cba-4f31-9a98-af75cabe90e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.716483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d25e6eba31e4159a938fb450270b6d0', 'request_id': 'req-0bc7a487-7cba-4f31-9a98-af75cabe90e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.723833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d25e6eba31e4159a938fb450270b6d0', 'request_id': 'req-0bc7a487-7cba-4f31-9a98-af75cabe90e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.738565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] The token cache is being invalidate because project 3d25e6eba31e4159a938fb450270b6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:48.739061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.739776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0bc7a487-7cba-4f31-9a98-af75cabe90e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:48.741394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 957/1885] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:48 2022] DELETE /identity/v3/projects/3d25e6eba31e4159a938fb450270b6d0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:48.746777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:48.747069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:48.747349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:48.766153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:48.767322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:48.832864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 929/1886] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:48.841764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:48.842102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:48.842480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:48.862449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:48.863858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:48.873409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 958/1887] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:48.879177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:48.891337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699eb55-7157-43ed-9635-9a4e5cbe35b8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:48.897450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:48.901123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:48.901464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:48.901750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:48.902652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:48.902972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:48.904759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:48.921295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 930/1888] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:48.926773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:48.944756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f45ac-0c02-4619-b5f9-b1bb536addc1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:48.952206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:48.952936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:48.953265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:48.953672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:48.954356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:48.957181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.038133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 959/1889] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:48 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:49.041997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.054081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da12788-99c4-4be7-8578-2690510a2bbf admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.138054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.138844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.139219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.140123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.140257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.142534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.195600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 931/1890] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:49 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:49.201187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.228239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1349-1bb5-44dc-abdb-feb1f6807520 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.247593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.248447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.248738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.249017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.250060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.251788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.274833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.277942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.298092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 960/1891] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:49.299358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.301362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46b1f6-2673-4fa2-be74-c01e8fd15cc7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.313536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3bc9c-894b-406d-aa71-84d24cc1a87e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.313864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.316187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.316769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.317431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] PATH_INFO: `/v3/users/9a5a1d934b1d42b2bf195b7c6183c632` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.319093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5a1d934b1d42b2bf195b7c6183c632)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.320245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.321856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.322158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.322687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.323169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.324132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.325922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.337245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:49.339591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 961/1892] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:49.345316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.365164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40311ae0-459b-4bd0-a0b9-547a6cf1d8a3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.372771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eec3fe-b039-47cb-b4dd-73dc498a5de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.373908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec3fe-b039-47cb-b4dd-73dc498a5de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.374438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec3fe-b039-47cb-b4dd-73dc498a5de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.374853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eec3fe-b039-47cb-b4dd-73dc498a5de5 admin admin] PATH_INFO: `/v3/projects/a2a0de9e5860439ea9de8beb3b741798/users/6f755d8819a546d7a66c28ce11557907/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.390641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:49.397926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eec3fe-b039-47cb-b4dd-73dc498a5de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a0de9e5860439ea9de8beb3b741798, user_id=6f755d8819a546d7a66c28ce11557907, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.402799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eec3fe-b039-47cb-b4dd-73dc498a5de5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.443744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 962/1893] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:49 2022] PUT /identity/v3/projects/a2a0de9e5860439ea9de8beb3b741798/users/6f755d8819a546d7a66c28ce11557907/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:49.453326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5a1d934b1d42b2bf195b7c6183c632'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:49.454381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.454790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.455145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.472739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5a1d934b1d42b2bf195b7c6183c632'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:49.497150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] MFA Rules not processed for user `6f755d8819a546d7a66c28ce11557907`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:49.498583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.521705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5a1d934b1d42b2bf195b7c6183c632'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:49.522506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5a1d934b1d42b2bf195b7c6183c632'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:49.524427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-17c715f5-0bed-44b3-bdc9-1dc1a456ab23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5a1d934b1d42b2bf195b7c6183c632'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:49.553307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 932/1894] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:49 2022] DELETE /identity/v3/users/9a5a1d934b1d42b2bf195b7c6183c632 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:49.562764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.563248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.563704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.587108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:49.589846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.610707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 963/1895] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:49.616078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.617918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.672557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 933/1896] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:49.683151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.684644 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:49.697444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760557-85c6-470e-948b-95fc071fd9b2 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.704446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.705145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.705436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.705722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.706432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.708376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.744886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 964/1897] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:49 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:49.749515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.778299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577582-4a34-4f23-9718-40459f70a54b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.847258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:49.851847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d03ef-b992-42ff-beb9-af781abef4a9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.853343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.854543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.854720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.855107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.857468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.858715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.859288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.859582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.860325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.862223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.867949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.869803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.873200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:49.881791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 934/1898] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:05:49.887559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 965/1899] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:49 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:49.891121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.901436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d649080-77b0-4e20-b46a-def9e075a44b service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.906660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.907151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.907434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.907709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.908435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.910659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.920686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 935/1900] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:05:49.924728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:49.935361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697398f-6182-4a72-aea5-1535c05a0898 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:49.952805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:49.954098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:49.954383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:49.954714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] PATH_INFO: `/v3/projects/3c5228d64ff14f708082d787bf30fb56/users/ede332ae8a2e40d2a2d6d94296432227/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:49.972560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5228d64ff14f708082d787bf30fb56, user_id=ede332ae8a2e40d2a2d6d94296432227, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:49.975105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:49.987672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 966/1901] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:49 2022] PUT /identity/v3/projects/3c5228d64ff14f708082d787bf30fb56/users/ede332ae8a2e40d2a2d6d94296432227/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:49.991618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.001712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193218c3-cb14-4df9-b14d-0b5835d1c1e5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:50.066427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93f095-ae25-4362-b66f-b66d4e73ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:50.067342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93f095-ae25-4362-b66f-b66d4e73ebe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.067762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93f095-ae25-4362-b66f-b66d4e73ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.067859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93f095-ae25-4362-b66f-b66d4e73ebe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.068602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93f095-ae25-4362-b66f-b66d4e73ebe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:50.070271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93f095-ae25-4362-b66f-b66d4e73ebe6 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:50.081573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 936/1902] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:50.086249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.097609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601cf8a-f8f6-4665-83a5-8ae5df4cce82 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:50.102525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:50.103482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.103765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.104036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] PATH_INFO: `/v3/projects/3c5228d64ff14f708082d787bf30fb56/users/ede332ae8a2e40d2a2d6d94296432227/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.106080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c5228d64ff14f708082d787bf30fb56, user_id=ede332ae8a2e40d2a2d6d94296432227, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:50.108767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:50.119539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 967/1903] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:50 2022] PUT /identity/v3/projects/3c5228d64ff14f708082d787bf30fb56/users/ede332ae8a2e40d2a2d6d94296432227/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:50.132008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.132401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.132780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.172439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] MFA Rules not processed for user `ede332ae8a2e40d2a2d6d94296432227`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:50.174909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.188595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.190359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.280629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 937/1904] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:50.291415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.292714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.302234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6800314-abad-4252-b460-c0730d651a61 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:50.307115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:50.308168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.308534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.308911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] PATH_INFO: `/v3/projects/c950aefee69f491eb643371682266e12` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.310184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c950aefee69f491eb643371682266e12)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:50.312198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:50.360059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184aa5-0966-4581-9235-d06645a66eb4 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:50.372588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:50.381841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c950aefee69f491eb643371682266e12', 'request_id': 'req-1a3d968e-45ee-4584-a164-28f653c6a224'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.388261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c950aefee69f491eb643371682266e12', 'request_id': 'req-1a3d968e-45ee-4584-a164-28f653c6a224'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.394557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c950aefee69f491eb643371682266e12', 'request_id': 'req-1a3d968e-45ee-4584-a164-28f653c6a224'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.406998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] The token cache is being invalidate because project c950aefee69f491eb643371682266e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:50.407285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.407940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.409776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 968/1905] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:50 2022] DELETE /identity/v3/projects/c950aefee69f491eb643371682266e12 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:50.410379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.411736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.413908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:50.414859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.415144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.415423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.417219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.475082 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:50.497882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:50.499496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:50.508564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d968e-45ee-4584-a164-28f653c6a224 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:50.513464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 938/1906] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:05:50.514272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:50.515056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.515300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.515592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.515871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] PATH_INFO: `/v3/users/2b8d88d1b386446b91146f8ff59b3e31` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.517107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8d88d1b386446b91146f8ff59b3e31)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:50.517669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.519666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:50.537932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92bd35-17e9-4f3b-9ceb-aa9425075893 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:50.552075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:50.553675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.554289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.554942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] PATH_INFO: `/v3/users/ca1ca412351148e1865196515525a295` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.556468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1ca412351148e1865196515525a295)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:50.558912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:50.614418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8d88d1b386446b91146f8ff59b3e31'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.624623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1ca412351148e1865196515525a295'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.629063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8d88d1b386446b91146f8ff59b3e31'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.643194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1ca412351148e1865196515525a295'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.657403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8d88d1b386446b91146f8ff59b3e31'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.659552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8d88d1b386446b91146f8ff59b3e31'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.661049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-899f13ce-6250-42f0-aaca-ae44708df9c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8d88d1b386446b91146f8ff59b3e31'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.671269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1ca412351148e1865196515525a295'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.673337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1ca412351148e1865196515525a295'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.674374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1ca412351148e1865196515525a295'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:50.694494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 969/1907] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:50 2022] DELETE /identity/v3/users/2b8d88d1b386446b91146f8ff59b3e31 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:50.698478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:50.698807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:50.699091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:50.701188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 939/1908] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:50 2022] DELETE /identity/v3/users/ca1ca412351148e1865196515525a295 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:50.757825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:50.759194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:50.977532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 970/1909] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:50.991049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:50.992530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:51.078392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:51.102069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a0ccc-4f17-40da-8d4b-0e7d64549d49 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:51.108479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:51.110073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:51.110921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:51.111417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:51.125944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:51.129050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:51.144054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 940/1910] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:51.590311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:51.590311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:51.602836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814488-b091-4ffe-83fc-856b9d0a0f5c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:51.618157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:51.619007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:51.619288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:51.620241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] PATH_INFO: `/v3/projects/ebe64309b53349f5acc1e7d352271a75` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:51.620889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe64309b53349f5acc1e7d352271a75)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:51.622844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:51.672335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe64309b53349f5acc1e7d352271a75', 'request_id': 'req-24deb12c-6b97-4284-9027-2316a8f5653b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.679528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe64309b53349f5acc1e7d352271a75', 'request_id': 'req-24deb12c-6b97-4284-9027-2316a8f5653b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.684463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:51.685652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe64309b53349f5acc1e7d352271a75', 'request_id': 'req-24deb12c-6b97-4284-9027-2316a8f5653b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.698466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] The token cache is being invalidate because project ebe64309b53349f5acc1e7d352271a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:51.698907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.699718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.701264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 971/1911] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:51 2022] DELETE /identity/v3/projects/ebe64309b53349f5acc1e7d352271a75 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:51.705405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:51.716539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e017f3e-f2e6-4b49-b07f-f84870addd67 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:51.777023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:51.777930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:51.778215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:51.778495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] PATH_INFO: `/v3/users/34a9cfaaa4d047048302cb6dd804ebd4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:51.779836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a9cfaaa4d047048302cb6dd804ebd4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:51.781786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:51.834094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a9cfaaa4d047048302cb6dd804ebd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.842336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a9cfaaa4d047048302cb6dd804ebd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.855243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a9cfaaa4d047048302cb6dd804ebd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.856130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a9cfaaa4d047048302cb6dd804ebd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.856739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6cd4b5ae-4d7d-4011-86d0-92840578e5cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a9cfaaa4d047048302cb6dd804ebd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:51.870240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 941/1912] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:51 2022] DELETE /identity/v3/users/34a9cfaaa4d047048302cb6dd804ebd4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:51.979611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:51.981858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.006900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb12c-6b97-4284-9027-2316a8f5653b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.085590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.086657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.087085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.087484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] PATH_INFO: `/v3/projects/a5b0be1610ef4763aaba135fb62c4229` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.089532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b0be1610ef4763aaba135fb62c4229)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.091730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.146292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b0be1610ef4763aaba135fb62c4229', 'request_id': 'req-34f15295-d936-4005-a07c-92c8696b37ec'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.152992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b0be1610ef4763aaba135fb62c4229', 'request_id': 'req-34f15295-d936-4005-a07c-92c8696b37ec'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.158809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b0be1610ef4763aaba135fb62c4229', 'request_id': 'req-34f15295-d936-4005-a07c-92c8696b37ec'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.172808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] The token cache is being invalidate because project a5b0be1610ef4763aaba135fb62c4229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:52.173107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.173869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.176298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 972/1913] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:51 2022] DELETE /identity/v3/projects/a5b0be1610ef4763aaba135fb62c4229 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:52.180984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ccd60-9e46-4347-a346-261ceaa321d6 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.181276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ccd60-9e46-4347-a346-261ceaa321d6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.181561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ccd60-9e46-4347-a346-261ceaa321d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.204917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b98ccd60-9e46-4347-a346-261ceaa321d6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:52.206200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b98ccd60-9e46-4347-a346-261ceaa321d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.215887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.217950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.309237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 942/1914] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:52.317339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.317595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.317866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.330718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:52.337351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:52.339107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.352826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 943/1915] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:52.353916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.357458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f15295-d936-4005-a07c-92c8696b37ec admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.363825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.363992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867be7-53b9-41f7-b530-d8bcb5af0108 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.366679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.366981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.367427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] PATH_INFO: `/v3/projects/5e8bd28e1cc644c3967e426164d7a964` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.368899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8bd28e1cc644c3967e426164d7a964)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.369251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.370077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.370507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.371003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.371178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.372048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:52.372500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.374217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.388664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 944/1916] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:52.393302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.403550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:52.409199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c33483-3984-4fe6-b685-35c8b6e1592d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.418989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.419979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.420281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.420569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.422022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.425252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.444989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8bd28e1cc644c3967e426164d7a964', 'request_id': 'req-744ce237-b977-4c9a-96c8-75d46b43312b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.454187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8bd28e1cc644c3967e426164d7a964', 'request_id': 'req-744ce237-b977-4c9a-96c8-75d46b43312b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.462695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8bd28e1cc644c3967e426164d7a964', 'request_id': 'req-744ce237-b977-4c9a-96c8-75d46b43312b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.470590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 945/1917] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:52.476306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.489787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] The token cache is being invalidate because project 5e8bd28e1cc644c3967e426164d7a964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:52.490557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.491563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-744ce237-b977-4c9a-96c8-75d46b43312b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:52.493795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 973/1918] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:52 2022] DELETE /identity/v3/projects/5e8bd28e1cc644c3967e426164d7a964 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:52.501614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628582-659e-41cf-8361-1572d0b2ba0b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.502580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628582-659e-41cf-8361-1572d0b2ba0b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.503153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628582-659e-41cf-8361-1572d0b2ba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.530763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e1628582-659e-41cf-8361-1572d0b2ba0b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:52.532929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e1628582-659e-41cf-8361-1572d0b2ba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.550645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c16266-ff80-402c-9608-105bf7e2d2e1 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.556216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.557002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.557319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.558902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.558902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.560372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.587743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 946/1919] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:52.591685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.594973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.608611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3453812-5ab0-4635-a9db-5cfb115673d8 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.614761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.615443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.615730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.616009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.617013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.618676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.621713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 974/1920] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:52.629859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.630225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.630581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.635182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 947/1921] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:05:52.636160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.646303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1eb44-3c29-44ba-ad51-7d32f370e09f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.651099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:52.651402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.652145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.652296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.652605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.652932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.653735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.655444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.660775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 975/1922] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:52.666907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.668413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 948/1923] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:52.672388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.677125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8cd1b-8c0c-47b9-b837-8625ca6fecc0 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.681809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.682449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.682763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.683039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.683159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a39de-3b98-4532-88dc-9eb9d7fe79fc admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.683767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:52.684070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.685656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.688480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6eb5e-d14a-4783-a799-f54883b9cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.689466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6eb5e-d14a-4783-a799-f54883b9cbac admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.689823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6eb5e-d14a-4783-a799-f54883b9cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.690177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6eb5e-d14a-4783-a799-f54883b9cbac admin admin] PATH_INFO: `/v3/projects/b65aed918f714bffb97bb5b79c6b0537/users/41482f309dd841b38cc2c870af1b811e/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.697428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 976/1924] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:05:52.701588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.707075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6eb5e-d14a-4783-a799-f54883b9cbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65aed918f714bffb97bb5b79c6b0537, user_id=41482f309dd841b38cc2c870af1b811e, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.709781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6eb5e-d14a-4783-a799-f54883b9cbac admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.711354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba8e37-7f6b-48c8-bbcc-36205b80140f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.716344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.717000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.717282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.717579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.718255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.720177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.734657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 949/1925] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:52 2022] PUT /identity/v3/projects/b65aed918f714bffb97bb5b79c6b0537/users/41482f309dd841b38cc2c870af1b811e/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:52.738133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.738763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.739350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.750960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 977/1926] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:52.755007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.790154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] MFA Rules not processed for user `41482f309dd841b38cc2c870af1b811e`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:52.792334 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:52.835442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c85149-d33b-4f5e-b485-a5726aac3a4e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.840360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.841234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.841597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.841951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.842764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.844801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.861448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:52.870419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 978/1927] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:52.874439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.886310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d65c9e-df87-428b-ac95-210d45599b2b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.890485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:52.891306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.892116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.892469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.892804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.894120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.895778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.910791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 979/1929] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:05:52.911018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 950/1929] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:52.913992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.915262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.924980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981d98c-ba8c-4ae6-8457-af166b17289b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.930351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.931038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.931325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.931606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.932323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.934213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.936115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3d3de-b44a-4d0c-91b7-78f4e808655c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:52.945835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:52.947578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:52.948078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:52.948582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:52.950021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:52.953100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:52.974924 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 951/1930] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:52.979150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.985445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 980/1931] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:52.989951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:52.995378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245d72b-e50a-424b-973b-8b6212d2de44 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.063191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fbd2f1-2842-44ad-a511-1f73f986073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.064176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbd2f1-2842-44ad-a511-1f73f986073b admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.064515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbd2f1-2842-44ad-a511-1f73f986073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.064808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fbd2f1-2842-44ad-a511-1f73f986073b admin admin] PATH_INFO: `/v3/projects/b33925410ec54e1c85163c26c95b9e14/users/8060ee5deacf427fbfb5b278d315d6db/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.067017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdb48d-0141-4199-a520-9702b8e30f74 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.073306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.073935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.074247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.074514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.075302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.077605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.091949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fbd2f1-2842-44ad-a511-1f73f986073b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33925410ec54e1c85163c26c95b9e14, user_id=8060ee5deacf427fbfb5b278d315d6db, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.095387 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fbd2f1-2842-44ad-a511-1f73f986073b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.102127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 981/1932] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:52 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:53.106815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.119735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 952/1933] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:52 2022] PUT /identity/v3/projects/b33925410ec54e1c85163c26c95b9e14/users/8060ee5deacf427fbfb5b278d315d6db/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:05:53.121839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.122161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.122472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.125601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f31219-8bc4-472f-a754-302a42f2089e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.151048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] MFA Rules not processed for user `8060ee5deacf427fbfb5b278d315d6db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:53.152284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:53.192143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.192864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.193148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.193420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.194142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.195922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.209571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 982/1934] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:53.219694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.232861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882fa0c-0188-4180-8697-3dc5df7654c2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.238548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 953/1935] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:53.241639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.241639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.241639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.241639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] PATH_INFO: `/v3/projects/81d586a840254e32aa8e971278b2b6a6/users/9fa630bf3fac447894fcdce6a06a2073/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.246725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.248190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:53.259390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d586a840254e32aa8e971278b2b6a6, user_id=9fa630bf3fac447894fcdce6a06a2073, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.262401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.295977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 983/1936] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:53 2022] PUT /identity/v3/projects/81d586a840254e32aa8e971278b2b6a6/users/9fa630bf3fac447894fcdce6a06a2073/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:53.299710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.326733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:53.344680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:53.345455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d9677-b383-46c4-9395-1b8ecf4528e0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.381891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f6fa-bae1-49e7-872f-12fcbdac0b72 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.386919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.387702 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.387897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.388151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.388890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.390492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.404528 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 984/1937] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:05:53.405543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.411627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.412288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.412574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.412851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.415616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbc484-f49e-4bb7-a280-41e970a5c754 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.423290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750f084-6408-4363-a16f-42b8d8d34e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.424176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f084-6408-4363-a16f-42b8d8d34e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.424459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f084-6408-4363-a16f-42b8d8d34e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.424951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750f084-6408-4363-a16f-42b8d8d34e20 admin admin] PATH_INFO: `/v3/projects/81d586a840254e32aa8e971278b2b6a6/users/9fa630bf3fac447894fcdce6a06a2073/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.427296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f084-6408-4363-a16f-42b8d8d34e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d586a840254e32aa8e971278b2b6a6, user_id=9fa630bf3fac447894fcdce6a06a2073, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.429933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750f084-6408-4363-a16f-42b8d8d34e20 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.446497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 985/1938] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:53 2022] PUT /identity/v3/projects/81d586a840254e32aa8e971278b2b6a6/users/9fa630bf3fac447894fcdce6a06a2073/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:53.453710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.454032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.454315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.486200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] MFA Rules not processed for user `9fa630bf3fac447894fcdce6a06a2073`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:53.486788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.487802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:53.488665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.501246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 954/1939] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:05:53.582233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 986/1940] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:53.594192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.596664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:53.700735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2b4e-255e-4adb-af24-c9e79c67e5ef service glance] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.764706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c808012-aa77-4f1a-af05-f1ac85a4e912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.765274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808012-aa77-4f1a-af05-f1ac85a4e912 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.765522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808012-aa77-4f1a-af05-f1ac85a4e912 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.765911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c808012-aa77-4f1a-af05-f1ac85a4e912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.780885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c808012-aa77-4f1a-af05-f1ac85a4e912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.782339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c808012-aa77-4f1a-af05-f1ac85a4e912 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.794680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 955/1941] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:53.825207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:53.901378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76bb9d-8f75-4602-bce3-73367ab158ac None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:53.909582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:53.910522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:53.910979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:53.911235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] PATH_INFO: `/v3/users/41482f309dd841b38cc2c870af1b811e` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:53.912489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41482f309dd841b38cc2c870af1b811e)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:53.914442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:53.960322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:53.982008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41482f309dd841b38cc2c870af1b811e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:53.995654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41482f309dd841b38cc2c870af1b811e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.008903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41482f309dd841b38cc2c870af1b811e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.009604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41482f309dd841b38cc2c870af1b811e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.010210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41482f309dd841b38cc2c870af1b811e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.050967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 987/1942] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:53 2022] DELETE /identity/v3/users/41482f309dd841b38cc2c870af1b811e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:54.055106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:54.055406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:54.055688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:54.106302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:54.107563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:54.111905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:54.196400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 956/1943] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:54.207898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:54.209400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:54.267918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3553e-8a75-4f74-8018-7c07f26b02d0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:54.272510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:54.273170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:54.274335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:54.274644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:54.285341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:54.286925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:54.297246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 988/1944] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:54.590494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:54.592001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:54.605363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfbce52-8f0a-428f-80b0-1af5499e9b66 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:54.614040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:54.615027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:54.615319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:54.615600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] PATH_INFO: `/v3/projects/b65aed918f714bffb97bb5b79c6b0537` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:54.616960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b65aed918f714bffb97bb5b79c6b0537)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:54.619100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:54.665971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b65aed918f714bffb97bb5b79c6b0537', 'request_id': 'req-a46c313e-7c6a-44f3-8179-10c0e7177532'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.671984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b65aed918f714bffb97bb5b79c6b0537', 'request_id': 'req-a46c313e-7c6a-44f3-8179-10c0e7177532'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.677290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b65aed918f714bffb97bb5b79c6b0537', 'request_id': 'req-a46c313e-7c6a-44f3-8179-10c0e7177532'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.689248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] The token cache is being invalidate because project b65aed918f714bffb97bb5b79c6b0537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:54.689532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.690167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.691725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 957/1945] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:54 2022] DELETE /identity/v3/projects/b65aed918f714bffb97bb5b79c6b0537 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:54.695666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:54.705568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b008b-d3b7-4b09-a4d1-c8572e0b1887 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:54.764543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:54.765329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:54.765611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:54.765888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] PATH_INFO: `/v3/users/9fa630bf3fac447894fcdce6a06a2073` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:54.767090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa630bf3fac447894fcdce6a06a2073)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:54.769663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:54.828442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa630bf3fac447894fcdce6a06a2073'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.844351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa630bf3fac447894fcdce6a06a2073'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.851348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:54.865209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa630bf3fac447894fcdce6a06a2073'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.867241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa630bf3fac447894fcdce6a06a2073'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.868844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-757a739f-9aae-494a-bb2e-0f4bcd183ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa630bf3fac447894fcdce6a06a2073'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:54.883849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 989/1946] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:05:54 2022] DELETE /identity/v3/users/9fa630bf3fac447894fcdce6a06a2073 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:55.203180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.204709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:55.218417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c313e-7c6a-44f3-8179-10c0e7177532 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.294181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.295081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.296187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.296187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] PATH_INFO: `/v3/projects/81d586a840254e32aa8e971278b2b6a6` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.297107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d586a840254e32aa8e971278b2b6a6)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.299388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.340205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:55.351432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d586a840254e32aa8e971278b2b6a6', 'request_id': 'req-bc431967-d091-4ce1-8d6f-22667692aac7'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:55.358104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d586a840254e32aa8e971278b2b6a6', 'request_id': 'req-bc431967-d091-4ce1-8d6f-22667692aac7'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:55.380409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d586a840254e32aa8e971278b2b6a6', 'request_id': 'req-bc431967-d091-4ce1-8d6f-22667692aac7'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:55.395728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] The token cache is being invalidate because project 81d586a840254e32aa8e971278b2b6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:05:55.396417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:55.396902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bc431967-d091-4ce1-8d6f-22667692aac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:05:55.398944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 958/1947] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:05:55 2022] DELETE /identity/v3/projects/81d586a840254e32aa8e971278b2b6a6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:55.403929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.404220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.404501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.425048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:55.426253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:55.499812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 990/1948] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:55.508239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.508876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.509195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.534826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:55.535971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:55.544293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 959/1949] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:55.548496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.558836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebdbb0-a09e-4f11-8aac-c06da5ef24a7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.564650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.565544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.565836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.566105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.566884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:05:55.568907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.570094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.586664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 991/1950] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:05:55.590517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.601367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78957a-6a71-4025-a428-5a1c3b2c90ee None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.605985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.606663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.606957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.607260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.608005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.609941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.639836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 960/1951] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:55 2022] POST /identity/v3/projects => generated 407 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:05:55.643936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.654664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd7ede-7730-44b4-9790-92c0c4887a3b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.755278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.755971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.756254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.756532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.757238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.759641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.785037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 992/1952] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:55 2022] POST /identity/v3/users => generated 304 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:55.789342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.800388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e2a4f-ec94-4ba4-9384-a71741e94e38 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.804966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.805619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.805900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.806175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.806934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.808514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.819044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 961/1953] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:55.822796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.832661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9ba48-4b46-4e60-896a-9ccc11e48a4f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.837288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978e91c-8205-4526-8914-baca6f04c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.837953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978e91c-8205-4526-8914-baca6f04c122 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.838241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978e91c-8205-4526-8914-baca6f04c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.838526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978e91c-8205-4526-8914-baca6f04c122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.839299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978e91c-8205-4526-8914-baca6f04c122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.840921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978e91c-8205-4526-8914-baca6f04c122 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.853006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 993/1954] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:55.857307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:55.868119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce900d-d187-4d59-a366-da83cd0f2a8b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:55.872711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:55.873579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.873855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.874126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] PATH_INFO: `/v3/projects/f9bb16e92b8148a28352cd26d7d7e992/users/cb2005696ac146bfb59a54bc12d5026a/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.888534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bb16e92b8148a28352cd26d7d7e992, user_id=cb2005696ac146bfb59a54bc12d5026a, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:55.891218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:55.905457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 962/1955] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:55 2022] PUT /identity/v3/projects/f9bb16e92b8148a28352cd26d7d7e992/users/cb2005696ac146bfb59a54bc12d5026a/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:55.916931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:55.917219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:55.917523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:55.944546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] MFA Rules not processed for user `cb2005696ac146bfb59a54bc12d5026a`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:55.945787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:55.984645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:56.034648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 994/1956] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:05:56.039122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.114159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40b1e-1f15-49fa-b087-f6320ea08ff0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.121599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.122286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.122571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.122882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.123583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.125626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.161251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 963/1957] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:05:56 2022] POST /identity/v3/projects => generated 407 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:56.174717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.193494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1eef3-e253-4fb4-aed6-c1ca5466df59 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.304761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.305669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.305952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.306235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.311962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.314155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.339638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 995/1958] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:05:56 2022] POST /identity/v3/users => generated 310 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:05:56.343817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.355923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2742b-013f-44a1-9d0e-6577afea8cec admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.361846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.362771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.363003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.363329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.364143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.365972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.384059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 964/1959] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:56.385565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.407466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f7313-330e-45e7-aeb8-2e0ff00118dc admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.414392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.414852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.415236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.415523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] PATH_INFO: `/v3/projects/67521725ccb44196b375c201b065eead/users/ee5fac71baff4213a03806ccfc5b6701/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.432072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67521725ccb44196b375c201b065eead, user_id=ee5fac71baff4213a03806ccfc5b6701, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.435205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.452938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 996/1960] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:56 2022] PUT /identity/v3/projects/67521725ccb44196b375c201b065eead/users/ee5fac71baff4213a03806ccfc5b6701/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:56.457874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.471530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89063ec-9ffc-4e9c-befe-d338e67ef301 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.519956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:05:56.577787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7342050-1280-4b91-b735-a037c34757bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.578490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7342050-1280-4b91-b735-a037c34757bf admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.578830 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7342050-1280-4b91-b735-a037c34757bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.579729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7342050-1280-4b91-b735-a037c34757bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.580038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7342050-1280-4b91-b735-a037c34757bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.582224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7342050-1280-4b91-b735-a037c34757bf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.597769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 965/1961] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:05:56.603407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.628433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a441f-22f3-4ff7-b13d-7bc34039c586 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.635783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.636979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.637316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.637565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] PATH_INFO: `/v3/projects/67521725ccb44196b375c201b065eead/users/ee5fac71baff4213a03806ccfc5b6701/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.639636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67521725ccb44196b375c201b065eead, user_id=ee5fac71baff4213a03806ccfc5b6701, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.643492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.658757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 997/1962] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:05:56 2022] PUT /identity/v3/projects/67521725ccb44196b375c201b065eead/users/ee5fac71baff4213a03806ccfc5b6701/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:05:56.669039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.669578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.670304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.699806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] MFA Rules not processed for user `ee5fac71baff4213a03806ccfc5b6701`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:05:56.700995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:56.769480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 966/1963] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:05:56 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:05:56.781611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:56.784542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:56.849724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c2b2-056f-4f64-8f98-79ed648fb1a2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:56.853433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:56.854084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:56.854371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:56.854680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:56.864094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:56.865513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:56.874948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 998/1964] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:56 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:05:57.073485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:05:57.076026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:05:57.091007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95594b89-713d-4c6f-86c3-81fab31ad999 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:05:57.094658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e2c91-75ca-481d-81f3-fb5b1634f58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:05:57.095309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2c91-75ca-481d-81f3-fb5b1634f58e service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:05:57.095593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2c91-75ca-481d-81f3-fb5b1634f58e service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:05:57.095867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e2c91-75ca-481d-81f3-fb5b1634f58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:05:57.155398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e2c91-75ca-481d-81f3-fb5b1634f58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:05:57.156907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e2c91-75ca-481d-81f3-fb5b1634f58e service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:05:57.166787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 967/1965] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:05:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:00.391543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:00.393568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:00.424522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:00.459828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913fcda-56e1-400e-a46a-884260675652 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:00.464323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:00.465190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:00.465477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:00.465755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] PATH_INFO: `/v3/users/63c04214c85e412c80d23fefd41ef4bb` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:00.467052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c04214c85e412c80d23fefd41ef4bb)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:00.468968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:00.515695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c04214c85e412c80d23fefd41ef4bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:00.523989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c04214c85e412c80d23fefd41ef4bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:00.547022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c04214c85e412c80d23fefd41ef4bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:00.548326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c04214c85e412c80d23fefd41ef4bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:00.548948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c04214c85e412c80d23fefd41ef4bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:00.563274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 999/1966] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:00 2022] DELETE /identity/v3/users/63c04214c85e412c80d23fefd41ef4bb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:00.573733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:00.573733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:00.573733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:00.591287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:00.592561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:00.631435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:00.632916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:00.647220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:00.660609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 968/1967] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:00.670258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:00.671650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:00.692748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf15e4-1f93-4c9b-85f1-edfecba52032 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:00.696618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:00.697696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:00.697986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:00.698297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:00.700501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:00.737335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31b976-7de1-4756-82f9-cc9740adbd9f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:00.741564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:00.742223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:00.742505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:00.742893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:00.752586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:00.754711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:00.765269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 969/1968] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:06:00.775681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:00.778410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:00.780762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:00.802865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1000/1969] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:06:01.165381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:01.166878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:01.180704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9405c-5d55-47ca-93f7-3364b80633a4 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:01.187564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:01.188530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:01.188819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:01.189098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] PATH_INFO: `/v3/projects/6908a9ea6abf4cadaa65a2690b525eba` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:01.190330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6908a9ea6abf4cadaa65a2690b525eba)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:01.193108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:01.246313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6908a9ea6abf4cadaa65a2690b525eba', 'request_id': 'req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.252446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6908a9ea6abf4cadaa65a2690b525eba', 'request_id': 'req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.258645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6908a9ea6abf4cadaa65a2690b525eba', 'request_id': 'req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.271296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] The token cache is being invalidate because project 6908a9ea6abf4cadaa65a2690b525eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:01.271647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.272388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.274313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 970/1970] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:01 2022] DELETE /identity/v3/projects/6908a9ea6abf4cadaa65a2690b525eba => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:01.278654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:01.289344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1a222-fe12-4aa9-9a79-8d445a430d59 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:01.345383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:01.346342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:01.346642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:01.346930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] PATH_INFO: `/v3/users/2b63e1bc4fec492f931ecc5d9ea68bf9` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:01.348131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b63e1bc4fec492f931ecc5d9ea68bf9)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:01.350195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:01.396118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b63e1bc4fec492f931ecc5d9ea68bf9'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.403155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b63e1bc4fec492f931ecc5d9ea68bf9'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.414927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b63e1bc4fec492f931ecc5d9ea68bf9'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.415553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b63e1bc4fec492f931ecc5d9ea68bf9'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.416154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-949cfa65-f5c0-4c4e-bc41-38ce72fc4795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b63e1bc4fec492f931ecc5d9ea68bf9'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.429036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1001/1971] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:01 2022] DELETE /identity/v3/users/2b63e1bc4fec492f931ecc5d9ea68bf9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:01.751790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:01.753286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:01.766090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d2459-9cd7-40a5-bb1b-ea6beae86d97 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:01.832095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:01.832974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:01.833260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:01.833561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] PATH_INFO: `/v3/projects/1ca748a4d3eb478aacf60ce011fa8efa` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:01.834773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca748a4d3eb478aacf60ce011fa8efa)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:01.836739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:01.859239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:01.883140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca748a4d3eb478aacf60ce011fa8efa', 'request_id': 'req-78e9a748-fca6-4b04-93d7-59e52818520c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.889094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca748a4d3eb478aacf60ce011fa8efa', 'request_id': 'req-78e9a748-fca6-4b04-93d7-59e52818520c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.895062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca748a4d3eb478aacf60ce011fa8efa', 'request_id': 'req-78e9a748-fca6-4b04-93d7-59e52818520c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.907182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] The token cache is being invalidate because project 1ca748a4d3eb478aacf60ce011fa8efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:01.907471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.908122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78e9a748-fca6-4b04-93d7-59e52818520c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:01.910489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 971/1972] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:01 2022] DELETE /identity/v3/projects/1ca748a4d3eb478aacf60ce011fa8efa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:01.915283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:01.915570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:01.915848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:01.936928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:01.938159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:01.998784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1002/1973] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:02.007210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.007600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.007941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.027183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:02.028294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:02.038100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 972/1974] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:02.042283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.052404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb8f7e-b5f9-4229-b191-63aaa14146fe None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.057171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.057902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.058257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.058646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.059546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:02.059850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.061451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.076123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1003/1975] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:02.078463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.088808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05116b-c18e-42fa-ba72-3dc083dbc8c9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.093724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.094347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.094656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.094943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.095643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.098202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.125252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 973/1976] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:02.129301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.134763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:02.141630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c86cf-6a7e-40a2-aef3-ca335c537fae admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.198774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.199440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.199726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.200000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.200695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.202600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.221805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1004/1977] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:02.226549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.236730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51164f3-37c0-45a1-a587-ee7ea78b7aad admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.241351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.242008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.242295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.242574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.243351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.244940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.255677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 974/1978] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:02.259437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.270004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a400e3f-ee43-4313-b418-1529d834f923 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.276744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c265f4-86c1-4022-b338-196cfb093cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.277565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c265f4-86c1-4022-b338-196cfb093cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.277803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c265f4-86c1-4022-b338-196cfb093cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.278083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c265f4-86c1-4022-b338-196cfb093cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.278958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c265f4-86c1-4022-b338-196cfb093cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.280450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c265f4-86c1-4022-b338-196cfb093cd6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.290705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1005/1979] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:02.294791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.304648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c16fe-7660-4c09-9e5e-95e3503cb002 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.309513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.310401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.310723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.311005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] PATH_INFO: `/v3/projects/42478ba0589e4c0f8d9d49f372c30513/users/13950efcd29347ebb69124ab4df1f105/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.326395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42478ba0589e4c0f8d9d49f372c30513, user_id=13950efcd29347ebb69124ab4df1f105, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.329089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.341985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 975/1980] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:02 2022] PUT /identity/v3/projects/42478ba0589e4c0f8d9d49f372c30513/users/13950efcd29347ebb69124ab4df1f105/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:02.352401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.352696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.352978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.380356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] MFA Rules not processed for user `13950efcd29347ebb69124ab4df1f105`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:02.381565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:02.445795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1006/1981] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:02.450189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.514167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65467db6-42ee-45f3-845d-80a694e6c2d1 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.519464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.520126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.520416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.520698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.521404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.523649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.550283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 976/1982] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:02.554305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.564745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fd040-8316-476b-a26e-00c71f499ec6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.620474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.621223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.621496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.621769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.622460 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.624389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.639963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:02.646160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1007/1983] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:02.649940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.660240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4d28e-3089-498a-b06a-506bff75f1c1 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.665130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.665790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.666080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.666352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.667107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.668709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.680503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 977/1984] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:02.684350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.694640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e9f1a-c410-411c-8136-b1a97138e5c5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.699117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.699976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.700255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.700526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] PATH_INFO: `/v3/projects/fba03f1e5d124d7eb931a8b312881bea/users/fb0876835ae24c4abeedf4b62a087a41/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.715148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba03f1e5d124d7eb931a8b312881bea, user_id=fb0876835ae24c4abeedf4b62a087a41, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.718177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.730361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1008/1985] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:02 2022] PUT /identity/v3/projects/fba03f1e5d124d7eb931a8b312881bea/users/fb0876835ae24c4abeedf4b62a087a41/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:02.734239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.744539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a2865-8b48-4ab5-8f20-655132a68d0e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.760974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:02.802147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa75818-887f-4b6b-b632-9367116ec7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.802779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa75818-887f-4b6b-b632-9367116ec7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.803067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa75818-887f-4b6b-b632-9367116ec7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.803367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa75818-887f-4b6b-b632-9367116ec7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.804100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa75818-887f-4b6b-b632-9367116ec7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.805765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa75818-887f-4b6b-b632-9367116ec7c9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.817264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 978/1986] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:02.821097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.831224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce605314-6bb4-4f42-9aea-c8c01f9a2e30 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:02.835772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:02.836651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.836936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.837209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] PATH_INFO: `/v3/projects/fba03f1e5d124d7eb931a8b312881bea/users/fb0876835ae24c4abeedf4b62a087a41/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.839261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba03f1e5d124d7eb931a8b312881bea, user_id=fb0876835ae24c4abeedf4b62a087a41, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:02.841875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:02.852661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1009/1987] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:02 2022] PUT /identity/v3/projects/fba03f1e5d124d7eb931a8b312881bea/users/fb0876835ae24c4abeedf4b62a087a41/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:02.863151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:02.863441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:02.863719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:02.890438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] MFA Rules not processed for user `fb0876835ae24c4abeedf4b62a087a41`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:02.891228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:02.967415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 979/1988] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:02.978761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:02.980525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:03.036977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644114bc-274b-49f6-a23d-f96336906d9b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:03.040558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:03.041193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:03.041480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:03.041762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:03.051461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:03.052932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:03.062181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1010/1989] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:03.133415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:03.142919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95885c72-6a69-472a-ad94-b11f86a22c86 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:03.146544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383af60-5f95-4f76-96f6-6a6de2e2b3d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:03.147219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383af60-5f95-4f76-96f6-6a6de2e2b3d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:03.147504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383af60-5f95-4f76-96f6-6a6de2e2b3d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:03.147784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383af60-5f95-4f76-96f6-6a6de2e2b3d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:03.209102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383af60-5f95-4f76-96f6-6a6de2e2b3d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:03.210549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383af60-5f95-4f76-96f6-6a6de2e2b3d1 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:03.219773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 980/1990] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:08.974239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:08.976922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:09.055503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14a85c-4094-4300-9e68-048f46c42f60 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:09.060361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:09.061415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:09.061748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:09.062109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] PATH_INFO: `/v3/users/cb2005696ac146bfb59a54bc12d5026a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:09.063549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2005696ac146bfb59a54bc12d5026a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:09.066579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:09.165646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2005696ac146bfb59a54bc12d5026a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.174536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2005696ac146bfb59a54bc12d5026a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.188167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:09.191651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2005696ac146bfb59a54bc12d5026a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.192658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2005696ac146bfb59a54bc12d5026a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.193761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2005696ac146bfb59a54bc12d5026a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.209314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1011/1991] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:08 2022] DELETE /identity/v3/users/cb2005696ac146bfb59a54bc12d5026a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:09.213998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:09.214439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:09.214567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:09.233833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:09.235222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:09.271604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:09.308406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 981/1992] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:09.319795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:09.321219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:09.395420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c59ac6-7203-4a78-aa9a-104a5a853406 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:09.400097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:09.400763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:09.401047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:09.401336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:09.411384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:09.412824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:09.424276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1012/1993] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:09.733916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:09.735453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:09.750163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244b069-e95d-4b69-9d4c-4117e59469bf None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:09.756166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:09.757063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:09.757350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:09.758412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] PATH_INFO: `/v3/projects/f9bb16e92b8148a28352cd26d7d7e992` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:09.760097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bb16e92b8148a28352cd26d7d7e992)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:09.761928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:09.810803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bb16e92b8148a28352cd26d7d7e992', 'request_id': 'req-afab9643-d5e0-41e7-8682-f4dd7a6110b4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.817790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bb16e92b8148a28352cd26d7d7e992', 'request_id': 'req-afab9643-d5e0-41e7-8682-f4dd7a6110b4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.823378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bb16e92b8148a28352cd26d7d7e992', 'request_id': 'req-afab9643-d5e0-41e7-8682-f4dd7a6110b4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.835872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] The token cache is being invalidate because project f9bb16e92b8148a28352cd26d7d7e992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:09.836100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.836716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.838343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 982/1994] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:09 2022] DELETE /identity/v3/projects/f9bb16e92b8148a28352cd26d7d7e992 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:09.842249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:09.852224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d9c0e-6b81-40f4-b64f-0ddd31de5ddc service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:09.911830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:09.912707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:09.912991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:09.913271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] PATH_INFO: `/v3/users/ee5fac71baff4213a03806ccfc5b6701` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:09.914514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee5fac71baff4213a03806ccfc5b6701)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:09.916504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:09.975073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee5fac71baff4213a03806ccfc5b6701'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:09.984497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee5fac71baff4213a03806ccfc5b6701'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.000683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee5fac71baff4213a03806ccfc5b6701'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.001436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee5fac71baff4213a03806ccfc5b6701'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.002177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-b2eee041-4516-4b68-bec1-4e96d7ace054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee5fac71baff4213a03806ccfc5b6701'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.017793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1013/1995] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:09 2022] DELETE /identity/v3/users/ee5fac71baff4213a03806ccfc5b6701 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:10.367002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.369989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:10.405778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab9643-d5e0-41e7-8682-f4dd7a6110b4 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.468255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.469129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.469409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.469688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] PATH_INFO: `/v3/projects/67521725ccb44196b375c201b065eead` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.470914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=67521725ccb44196b375c201b065eead)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.473500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:10.519925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67521725ccb44196b375c201b065eead', 'request_id': 'req-2e14b8cd-0571-4b8d-af05-005f32e876ad'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.526335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67521725ccb44196b375c201b065eead', 'request_id': 'req-2e14b8cd-0571-4b8d-af05-005f32e876ad'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.532206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67521725ccb44196b375c201b065eead', 'request_id': 'req-2e14b8cd-0571-4b8d-af05-005f32e876ad'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.535308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:10.547591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] The token cache is being invalidate because project 67521725ccb44196b375c201b065eead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:10.547981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.549031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2e14b8cd-0571-4b8d-af05-005f32e876ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:10.551001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 983/1996] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:10 2022] DELETE /identity/v3/projects/67521725ccb44196b375c201b065eead => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:10.555850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.556171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.556452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.569477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:10.575970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:10.577799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:10.639685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1014/1997] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:10.648287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.648577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.648854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.667315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:10.668446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:10.676550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 984/1998] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:10.680959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.691139 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816e3bd-1f20-4e46-afca-0a936ee2473d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.695836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.696498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.696785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.697066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.697795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:10.698107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.699749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:10.713330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1015/1999] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:06:10.717765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.727084 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff4b2d-98fc-4bb9-9cc4-92c1ffdb91cb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.733883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.734943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.735428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.735946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.736912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.739022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:10.774110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 985/2000] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:10 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:10.778219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.788670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f88d68-7622-4458-baaa-b6b4511ce903 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.854739 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.855534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.855887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.856244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.856948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.858898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:10.880655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1016/2001] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:10 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:06:10.885156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.896488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef7c9c-7d4d-42af-9a55-75ff9d1d30d0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.901635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.902316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.902603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.902912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.903653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.905261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:10.916294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 986/2002] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:10.920715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.932006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfedb68-4942-4fa6-b0b1-cd859d76ce7a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.936639 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de3978-2fe7-46c5-af5e-33826bc2472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.937303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de3978-2fe7-46c5-af5e-33826bc2472a admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.937595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de3978-2fe7-46c5-af5e-33826bc2472a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.937873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de3978-2fe7-46c5-af5e-33826bc2472a admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.938605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de3978-2fe7-46c5-af5e-33826bc2472a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.940787 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de3978-2fe7-46c5-af5e-33826bc2472a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:10.951233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1017/2003] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:10.955275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:10.966183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b3b16-c2fc-4c12-9b06-c0793eec8e9a admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:10.970769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:10.971734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:10.972021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:10.972300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] PATH_INFO: `/v3/projects/37ac67a4650e465ca7f9982ad188dc34/users/c6e129262a824c0594101fc7f85775e7/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:10.988973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ac67a4650e465ca7f9982ad188dc34, user_id=c6e129262a824c0594101fc7f85775e7, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:10.991700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:11.005563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 987/2004] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:10 2022] PUT /identity/v3/projects/37ac67a4650e465ca7f9982ad188dc34/users/c6e129262a824c0594101fc7f85775e7/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:11.016596 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:11.017163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:11.017636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:11.047211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] MFA Rules not processed for user `c6e129262a824c0594101fc7f85775e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:11.048389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:11.140840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1018/2005] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:11.152492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:11.154210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:11.251959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d57c1f-61d2-4739-b15e-046be24cd2ec admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:11.256262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecec44-7696-45bf-943f-1e1dd1c6cdeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:11.256930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecec44-7696-45bf-943f-1e1dd1c6cdeb service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:11.257213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecec44-7696-45bf-943f-1e1dd1c6cdeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:11.257490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecec44-7696-45bf-943f-1e1dd1c6cdeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:11.273328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecec44-7696-45bf-943f-1e1dd1c6cdeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:11.274759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecec44-7696-45bf-943f-1e1dd1c6cdeb service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:11.285588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 988/2006] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:06:11.315012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:11.319434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:11.384370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb8090-5b52-425f-a2c0-53aac1431782 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:11.388890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:11.389767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:11.390052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:11.390349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] PATH_INFO: `/v3/users/c6e129262a824c0594101fc7f85775e7` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:11.391592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e129262a824c0594101fc7f85775e7)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:11.393548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:11.444878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e129262a824c0594101fc7f85775e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:11.455089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e129262a824c0594101fc7f85775e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:11.467688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e129262a824c0594101fc7f85775e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:11.469518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e129262a824c0594101fc7f85775e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:11.470202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e129262a824c0594101fc7f85775e7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:11.482970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1019/2007] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:11 2022] DELETE /identity/v3/users/c6e129262a824c0594101fc7f85775e7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:11.488164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:11.488627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:11.488854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:11.506928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:11.508130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:11.569775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 989/2008] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:11.579284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:11.581209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:11.640742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07570a-8b6b-469a-ab9c-3a3e303213ce admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:11.644933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254310f-84a3-4100-8990-fc3844d6689c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:11.645586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254310f-84a3-4100-8990-fc3844d6689c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:11.645870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254310f-84a3-4100-8990-fc3844d6689c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:11.646153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254310f-84a3-4100-8990-fc3844d6689c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:11.655839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254310f-84a3-4100-8990-fc3844d6689c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:11.657510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254310f-84a3-4100-8990-fc3844d6689c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:11.670336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1020/2009] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:11.956957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:11.958498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:11.973936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fad2bf-d345-4652-b2a4-322d0a74ed77 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:11.979237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:11.980111 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:11.980455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:11.980759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] PATH_INFO: `/v3/projects/37ac67a4650e465ca7f9982ad188dc34` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:11.981958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ac67a4650e465ca7f9982ad188dc34)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:11.983985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.017991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:12.039961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ac67a4650e465ca7f9982ad188dc34', 'request_id': 'req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:12.046868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ac67a4650e465ca7f9982ad188dc34', 'request_id': 'req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:12.053106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ac67a4650e465ca7f9982ad188dc34', 'request_id': 'req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:12.073634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] The token cache is being invalidate because project 37ac67a4650e465ca7f9982ad188dc34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:12.074029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:12.074688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a221e46a-5ccb-41ca-8c94-6721e9d7d6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:12.077893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 990/2010] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:11 2022] DELETE /identity/v3/projects/37ac67a4650e465ca7f9982ad188dc34 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:12.083428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.083829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.084033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.102902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:12.104126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:12.139090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:12.185182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1021/2011] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:12.193521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.193812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.194097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.213424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:12.214763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:12.223217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 991/2012] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:12.227516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.238465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de500-d844-48ed-afe2-a431e5c4532a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.243401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.244097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.244429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.244735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.245490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:12.245807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.247556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.260315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1022/2013] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:12.264557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.280348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db833f9a-977b-48a9-b449-889ee6522e7d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.285445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.286138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.286419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.286728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.287452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.289465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.317036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 992/2014] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:12 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:12.323926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.334525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b02fe0-9621-4ec3-8395-dbb16b596ed2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.391129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.391806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.392094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.392371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.393067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.395072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.414454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1023/2015] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:12 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:12.418434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.429397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1efbf8-8ec6-410c-a778-1d0aa198def2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.434021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.434784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.435079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.435356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.436116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.437700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.448626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 993/2016] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:12.452321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.462710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07c841-cef6-4166-b8f3-1fdf47e6098e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.468609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07410981-b14c-45ce-8cf2-9219b3ea2317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.469334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410981-b14c-45ce-8cf2-9219b3ea2317 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.469623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410981-b14c-45ce-8cf2-9219b3ea2317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.469900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07410981-b14c-45ce-8cf2-9219b3ea2317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.470707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410981-b14c-45ce-8cf2-9219b3ea2317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.472315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07410981-b14c-45ce-8cf2-9219b3ea2317 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.482815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1024/2017] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:12.487376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.497300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294debb-2e99-420c-a695-4f3b480899cc admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.502001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.503554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.503837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.504115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] PATH_INFO: `/v3/projects/992945dc7186465d8969d3013feed074/users/3a2912f203e44d1ab0162fbc205db0b3/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.518311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=992945dc7186465d8969d3013feed074, user_id=3a2912f203e44d1ab0162fbc205db0b3, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.521023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.534137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 994/2018] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:12 2022] PUT /identity/v3/projects/992945dc7186465d8969d3013feed074/users/3a2912f203e44d1ab0162fbc205db0b3/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:12.546606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.546924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.547200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.573943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] MFA Rules not processed for user `3a2912f203e44d1ab0162fbc205db0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:12.575162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:12.639159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1025/2019] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:12.650069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:12.651793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:12.709523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09879bd2-b9e3-41b1-b4cd-fd0fbe015c8d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:12.713205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131533d-c52f-4fdb-8db1-f6c0ff75bf8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:12.713868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131533d-c52f-4fdb-8db1-f6c0ff75bf8f service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:12.714153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131533d-c52f-4fdb-8db1-f6c0ff75bf8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:12.714435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131533d-c52f-4fdb-8db1-f6c0ff75bf8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:12.723795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131533d-c52f-4fdb-8db1-f6c0ff75bf8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:12.725390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131533d-c52f-4fdb-8db1-f6c0ff75bf8f service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:12.734986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 995/2020] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:16.493317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:16.494969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:16.521184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:16.583262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d581342-c941-4e8b-92c6-0f23c6b614b9 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:16.588019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:16.588985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:16.589467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:16.589751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] PATH_INFO: `/v3/users/6f755d8819a546d7a66c28ce11557907` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:16.591006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f755d8819a546d7a66c28ce11557907)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:16.592926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:16.639829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f755d8819a546d7a66c28ce11557907'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:16.647379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f755d8819a546d7a66c28ce11557907'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:16.659395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f755d8819a546d7a66c28ce11557907'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:16.660053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f755d8819a546d7a66c28ce11557907'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:16.660836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f755d8819a546d7a66c28ce11557907'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:16.673987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1026/2021] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:16 2022] DELETE /identity/v3/users/6f755d8819a546d7a66c28ce11557907 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:16.678588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:16.679085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:16.679181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:16.696702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:16.697879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:16.744013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:16.757195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 996/2022] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:16.766554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:16.768100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:16.828902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215c239-6d9a-41fa-8e8e-b0c5b1840480 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:16.834923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:16.835620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:16.835933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:16.836274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:16.846852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:16.848294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:16.858338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1027/2023] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:17.290763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:17.292273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:17.305350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadce21-c042-4a3e-9271-6f5d08a15aa3 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:17.310362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:17.311356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:17.311710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:17.312061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] PATH_INFO: `/v3/projects/a2a0de9e5860439ea9de8beb3b741798` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:17.313353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a0de9e5860439ea9de8beb3b741798)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:17.315493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:17.364717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a0de9e5860439ea9de8beb3b741798', 'request_id': 'req-5fe77df2-b14a-4edc-9252-48ab22c4a98f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.371051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a0de9e5860439ea9de8beb3b741798', 'request_id': 'req-5fe77df2-b14a-4edc-9252-48ab22c4a98f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.376517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a0de9e5860439ea9de8beb3b741798', 'request_id': 'req-5fe77df2-b14a-4edc-9252-48ab22c4a98f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.389028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] The token cache is being invalidate because project a2a0de9e5860439ea9de8beb3b741798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:17.389322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.389963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.391598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 997/2024] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:17 2022] DELETE /identity/v3/projects/a2a0de9e5860439ea9de8beb3b741798 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:17.395544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:17.406004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08334452-be4f-46f6-b407-e12b7d3f24d2 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:17.452756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:17.479093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:17.480217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:17.481142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:17.481431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] PATH_INFO: `/v3/users/ede332ae8a2e40d2a2d6d94296432227` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:17.483048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede332ae8a2e40d2a2d6d94296432227)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:17.485291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:17.539026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede332ae8a2e40d2a2d6d94296432227'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.547689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede332ae8a2e40d2a2d6d94296432227'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.565658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede332ae8a2e40d2a2d6d94296432227'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.566402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede332ae8a2e40d2a2d6d94296432227'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.567039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-d31c117e-4ebc-4daa-93e9-a69f7a0e5225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede332ae8a2e40d2a2d6d94296432227'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:17.583338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1028/2025] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:17 2022] DELETE /identity/v3/users/ede332ae8a2e40d2a2d6d94296432227 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:17.883692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:17.888299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:17.904122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77df2-b14a-4edc-9252-48ab22c4a98f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:17.964072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:17.964964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:17.965250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:17.965526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] PATH_INFO: `/v3/projects/3c5228d64ff14f708082d787bf30fb56` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:17.966793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c5228d64ff14f708082d787bf30fb56)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:17.968722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.015852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c5228d64ff14f708082d787bf30fb56', 'request_id': 'req-e61fafcb-02a1-4fea-9ea9-7cae9c338631'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.022488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c5228d64ff14f708082d787bf30fb56', 'request_id': 'req-e61fafcb-02a1-4fea-9ea9-7cae9c338631'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.028234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c5228d64ff14f708082d787bf30fb56', 'request_id': 'req-e61fafcb-02a1-4fea-9ea9-7cae9c338631'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.040463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] The token cache is being invalidate because project 3c5228d64ff14f708082d787bf30fb56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:18.040785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.041418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e61fafcb-02a1-4fea-9ea9-7cae9c338631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.043014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 998/2026] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:17 2022] DELETE /identity/v3/projects/3c5228d64ff14f708082d787bf30fb56 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:18.048979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.049270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.049549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.067545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:18.068761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:18.126966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1029/2027] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:18.135711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.136008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.136296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.155145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:18.156362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:18.164443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 999/2028] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:18.168570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.178224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84af81-899c-4ade-bb65-1d8525d9802c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.182786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.183426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.183707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.183981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.184701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:18.185036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.186668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.198818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1030/2029] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:18.203005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.208405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:18.214604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0fd92-190a-496e-a28b-6716b9ee3a54 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.220700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.221370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.221657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.221940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.222663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.224604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.249549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1000/2030] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:18 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:18.254298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.264453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f5890-a144-4244-b196-4af84b686db0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.311110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:18.324406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.325069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.325350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.325629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.326321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.328250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.347269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1031/2031] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:18 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:18.351450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.361451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317f8ec-0b91-459f-85e1-eeb2a7499105 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.366449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.367517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.367798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.368067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.368789 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.370370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.381126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1001/2032] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:18.384723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.394395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26c193-4b7c-42de-a3e8-b353fab622fd admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.399402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44debb38-3773-4e8c-91b6-099af47ec5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.400052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44debb38-3773-4e8c-91b6-099af47ec5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.400338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44debb38-3773-4e8c-91b6-099af47ec5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.400618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44debb38-3773-4e8c-91b6-099af47ec5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.401344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44debb38-3773-4e8c-91b6-099af47ec5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.403021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44debb38-3773-4e8c-91b6-099af47ec5d5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.412907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1032/2033] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:18.416782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.426782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57f7e1-79dd-43a5-b977-a23fe9729f87 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.431398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.432263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.432582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.432861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] PATH_INFO: `/v3/projects/d1bf916234dd4da59c9eb155ede5f21e/users/9de6a2525d3c4d3c84218bbf72aeb853/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.449297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1bf916234dd4da59c9eb155ede5f21e, user_id=9de6a2525d3c4d3c84218bbf72aeb853, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.451980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.464764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1002/2034] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:18 2022] PUT /identity/v3/projects/d1bf916234dd4da59c9eb155ede5f21e/users/9de6a2525d3c4d3c84218bbf72aeb853/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:18.475851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.476146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.476515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.506916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] MFA Rules not processed for user `9de6a2525d3c4d3c84218bbf72aeb853`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:18.508428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:18.574719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1033/2035] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:18.604325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.606006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:18.663907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c0fb-9620-4de5-805f-286d320b1ab6 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.667588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f56d1-8921-47f6-961b-ac18a62c1d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.668181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f56d1-8921-47f6-961b-ac18a62c1d9c service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.668461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f56d1-8921-47f6-961b-ac18a62c1d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.668735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f56d1-8921-47f6-961b-ac18a62c1d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.678489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f56d1-8921-47f6-961b-ac18a62c1d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.680024 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f56d1-8921-47f6-961b-ac18a62c1d9c service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.689022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1003/2036] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:18.703741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.765505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a25d0-a9f2-445a-a09c-821bde169160 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:18.770141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:18.771045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.771329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.771607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] PATH_INFO: `/v3/users/9de6a2525d3c4d3c84218bbf72aeb853` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.773051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de6a2525d3c4d3c84218bbf72aeb853)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:18.775001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:18.820607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de6a2525d3c4d3c84218bbf72aeb853'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.827092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de6a2525d3c4d3c84218bbf72aeb853'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.831709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:18.841355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de6a2525d3c4d3c84218bbf72aeb853'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.842146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de6a2525d3c4d3c84218bbf72aeb853'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.842809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de6a2525d3c4d3c84218bbf72aeb853'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:18.855177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1034/2037] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:18 2022] DELETE /identity/v3/users/9de6a2525d3c4d3c84218bbf72aeb853 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:18.859759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:18.860078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:18.860355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:18.881305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:18.883496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:18.948219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1004/2038] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:18.957717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:18.959230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:19.022469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fb5a7-cb0d-4efb-b892-459261007eca admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.026358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79724825-1c06-4cac-a94a-2b24a9c884ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.027045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79724825-1c06-4cac-a94a-2b24a9c884ad service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.027328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79724825-1c06-4cac-a94a-2b24a9c884ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.027606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79724825-1c06-4cac-a94a-2b24a9c884ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.037449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79724825-1c06-4cac-a94a-2b24a9c884ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.038947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79724825-1c06-4cac-a94a-2b24a9c884ad service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.048538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1035/2039] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:19.481739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.483278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:19.495966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6184e-a9e8-4831-bfbc-e05fb8580b7d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.500573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.501442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.501722 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.501997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] PATH_INFO: `/v3/projects/d1bf916234dd4da59c9eb155ede5f21e` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.503804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1bf916234dd4da59c9eb155ede5f21e)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.505779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.549725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:19.554844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1bf916234dd4da59c9eb155ede5f21e', 'request_id': 'req-465c9523-56a3-4d89-a92a-102156d361ba'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:19.562476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1bf916234dd4da59c9eb155ede5f21e', 'request_id': 'req-465c9523-56a3-4d89-a92a-102156d361ba'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:19.570544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1bf916234dd4da59c9eb155ede5f21e', 'request_id': 'req-465c9523-56a3-4d89-a92a-102156d361ba'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:19.584270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] The token cache is being invalidate because project d1bf916234dd4da59c9eb155ede5f21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:19.584795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:19.585436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-465c9523-56a3-4d89-a92a-102156d361ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:19.587262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1005/2040] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:19 2022] DELETE /identity/v3/projects/d1bf916234dd4da59c9eb155ede5f21e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:19.592062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.592535 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.592641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.610014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:19.611258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:19.669034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1036/2041] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:19.677648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.677935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.678215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.696587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:19.697779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:19.706199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1006/2042] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:19.710158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.719816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca311f-3eaa-4103-8b08-4e5258768f59 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.724387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.725032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.725318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.725595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.726338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:19.726666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.728244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.739774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1037/2043] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:19.743530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.753316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842fbe0-d37c-4fbd-8032-af47cd732424 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.757934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.758593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.758906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.759184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.759887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.761786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.792084 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1007/2044] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:19 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:19.793446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.804682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d521-3fd7-4cb1-8584-5e541bb7b3e3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.859078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.859744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.860204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.860490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.861178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.863648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.872900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:19.885059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1038/2045] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:19 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:19.889302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.899679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aa9a6-a425-498c-a2c1-996d3fdb1f55 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.904402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.905066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.905352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.905630 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.906390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.908068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.918465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1008/2046] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:19.922117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.932617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae53ae-6d5c-4669-9f8f-5101fe312f99 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.937265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad04e5-7f5e-40ba-80d1-a9fc051f1fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.938021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad04e5-7f5e-40ba-80d1-a9fc051f1fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.938348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad04e5-7f5e-40ba-80d1-a9fc051f1fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.938687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad04e5-7f5e-40ba-80d1-a9fc051f1fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.939414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad04e5-7f5e-40ba-80d1-a9fc051f1fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.941628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad04e5-7f5e-40ba-80d1-a9fc051f1fb1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:19.951364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1039/2047] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:19.955326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:19.965109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b130433-4a3c-499b-8684-efc773ccc81c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:19.969691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:19.970559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:19.970880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:19.971157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] PATH_INFO: `/v3/projects/651b2ebb2a41444788a568ee990fd2c2/users/bd29b7d9022e4abd9dee0c596c310ee8/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:19.985385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=651b2ebb2a41444788a568ee990fd2c2, user_id=bd29b7d9022e4abd9dee0c596c310ee8, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:19.988551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.001388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1009/2048] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:19 2022] PUT /identity/v3/projects/651b2ebb2a41444788a568ee990fd2c2/users/bd29b7d9022e4abd9dee0c596c310ee8/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:20.011969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.012284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.012569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.038564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] MFA Rules not processed for user `bd29b7d9022e4abd9dee0c596c310ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:20.039945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:20.105111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1040/2049] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:20.109619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.175088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c18ce0-3275-4dc5-8d89-8b9ee4e06705 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.180146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.180667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.180784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.181019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.181714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.183693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.210654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1010/2050] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:20.214782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.227485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01deb2-38a8-4730-a2cf-8b15704e4414 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.285670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.286338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.286636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.286951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.287670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.289614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.309540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1041/2051] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:06:20.313681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.323705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad38c1c-80c5-443f-82ac-886345d7d72e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.328265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.328862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.329141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.329415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.330162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.331802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.342072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1011/2052] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:20.345878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.356224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08438fcf-bbe7-47ca-ad2e-7d30a79cec2d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.360827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b1edc-cdcd-4173-9c13-9dfa19018876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.361473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b1edc-cdcd-4173-9c13-9dfa19018876 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.362401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b1edc-cdcd-4173-9c13-9dfa19018876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.362720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b1edc-cdcd-4173-9c13-9dfa19018876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.363449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b1edc-cdcd-4173-9c13-9dfa19018876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.365025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b1edc-cdcd-4173-9c13-9dfa19018876 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.374780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1042/2053] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:20.378509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.388048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964d1a-1685-421d-8a36-4f2cfced7b30 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.392605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.393484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.393765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.394038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] PATH_INFO: `/v3/projects/cb35e650441e45c383f29929052a7a00/users/bb1088aad76540678fe83c101ead44f3/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.408368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb35e650441e45c383f29929052a7a00, user_id=bb1088aad76540678fe83c101ead44f3, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.411554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.423829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1012/2054] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:20 2022] PUT /identity/v3/projects/cb35e650441e45c383f29929052a7a00/users/bb1088aad76540678fe83c101ead44f3/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:20.434094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.434417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.435008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.460554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] MFA Rules not processed for user `bb1088aad76540678fe83c101ead44f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:20.461747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:20.502754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:20.526474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1043/2055] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:20.530957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.551561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:20.594473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569cda3-1ceb-4b35-b547-e184f048e700 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.599812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.600466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.600749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.601022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.601709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.603626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.629185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1013/2056] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:20.633168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.645190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d973d4-e1b3-46f1-80ad-f3b776de2147 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.700057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.700715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.701003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.701277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.701952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.703869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.723610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1044/2057] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:20.727535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.737823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2f7be-c573-4989-82df-3abfa91d0426 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.742446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.743263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.743549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.743832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.744565 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.746168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.756713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1014/2058] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:20.760795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.771145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627f950-b516-4918-95dc-d4ef7d741e41 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.776086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.776963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.777275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.777583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] PATH_INFO: `/v3/projects/bdc3fd92e43b40289bbb89a4be5b8247/users/4cb2230b1c2342b4be5cbd572cc2a244/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.792777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc3fd92e43b40289bbb89a4be5b8247, user_id=4cb2230b1c2342b4be5cbd572cc2a244, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.795490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.808346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1045/2059] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:20 2022] PUT /identity/v3/projects/bdc3fd92e43b40289bbb89a4be5b8247/users/4cb2230b1c2342b4be5cbd572cc2a244/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:20.812081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.822031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f92924-78cd-43a6-bffb-dbdb02a0fe73 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.879252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810835-4de0-41be-9a69-fe59334fc0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.879925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810835-4de0-41be-9a69-fe59334fc0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.880210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810835-4de0-41be-9a69-fe59334fc0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.880488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810835-4de0-41be-9a69-fe59334fc0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.881215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c810835-4de0-41be-9a69-fe59334fc0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.882836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c810835-4de0-41be-9a69-fe59334fc0cd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.892395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1015/2060] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:20.896858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:20.907366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d115-ec11-4ada-8999-3455532eb84b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:20.912641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:20.913516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.913801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.914110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] PATH_INFO: `/v3/projects/bdc3fd92e43b40289bbb89a4be5b8247/users/4cb2230b1c2342b4be5cbd572cc2a244/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.916130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc3fd92e43b40289bbb89a4be5b8247, user_id=4cb2230b1c2342b4be5cbd572cc2a244, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:20.918822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:20.928940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1046/2061] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:20 2022] PUT /identity/v3/projects/bdc3fd92e43b40289bbb89a4be5b8247/users/4cb2230b1c2342b4be5cbd572cc2a244/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:20.939643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:20.939937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:20.940216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:20.965955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] MFA Rules not processed for user `4cb2230b1c2342b4be5cbd572cc2a244`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:20.967173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:21.033320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1016/2062] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:21.044841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:21.046555 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:21.103328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac375b-594d-46ad-aa18-a9d94e1b6c44 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:21.106927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:21.107568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:21.107855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:21.108132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:21.154465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:21.165232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:21.166722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:21.175543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1047/2063] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:23.716365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:23.727117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0d275-7bb6-46fa-961d-2bdc39a5ca39 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:23.730771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:23.731420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:23.731701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:23.731973 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:23.741396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:23.794357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:23.795820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:23.804813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1017/2064] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:26.399742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:26.408030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394a8c1-f922-45e2-b55c-4cf60932b00f service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:26.412142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b31a2-79c8-4917-bbb2-4e442fe01084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:26.412807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b31a2-79c8-4917-bbb2-4e442fe01084 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:26.413097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b31a2-79c8-4917-bbb2-4e442fe01084 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:26.413373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b31a2-79c8-4917-bbb2-4e442fe01084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:26.422638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b31a2-79c8-4917-bbb2-4e442fe01084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:26.424097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b31a2-79c8-4917-bbb2-4e442fe01084 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:26.434467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1048/2065] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:30.793786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:30.795328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:30.867875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e616a02-cc34-43d9-a2b6-232d5e4b1e82 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:30.872739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:30.873773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:30.874053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:30.874325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] PATH_INFO: `/v3/users/13950efcd29347ebb69124ab4df1f105` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:30.875549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] RBAC: Authorizing `identity:delete_user(user_id=13950efcd29347ebb69124ab4df1f105)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:30.877485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:30.937537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13950efcd29347ebb69124ab4df1f105'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:30.946270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13950efcd29347ebb69124ab4df1f105'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:30.963554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13950efcd29347ebb69124ab4df1f105'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:30.964432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13950efcd29347ebb69124ab4df1f105'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:30.965086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13950efcd29347ebb69124ab4df1f105'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:30.981327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1018/2066] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:30 2022] DELETE /identity/v3/users/13950efcd29347ebb69124ab4df1f105 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:30.984582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:30.985025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:30.985173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:31.004513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:31.005736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:31.065074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1049/2067] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:31.075072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:31.077284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:31.115544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:31.144227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf3914-7f41-4fde-a944-a5ce2f80fbec admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:31.148566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:31.149340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:31.149681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:31.150022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:31.165918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:31.167488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:31.176797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1019/2068] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:31.435996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:31.437451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:31.451046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f9a7f-0d2e-4b7a-b68d-19f81128af7f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:31.458027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:31.459613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:31.460118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:31.460781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] PATH_INFO: `/v3/projects/42478ba0589e4c0f8d9d49f372c30513` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:31.462898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42478ba0589e4c0f8d9d49f372c30513)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:31.467092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:31.513760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42478ba0589e4c0f8d9d49f372c30513', 'request_id': 'req-5cb25272-ee2d-40a0-baa6-280b36347572'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.520519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42478ba0589e4c0f8d9d49f372c30513', 'request_id': 'req-5cb25272-ee2d-40a0-baa6-280b36347572'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.526181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42478ba0589e4c0f8d9d49f372c30513', 'request_id': 'req-5cb25272-ee2d-40a0-baa6-280b36347572'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.537729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] The token cache is being invalidate because project 42478ba0589e4c0f8d9d49f372c30513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:31.538012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.538709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.540158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1050/2069] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:31 2022] DELETE /identity/v3/projects/42478ba0589e4c0f8d9d49f372c30513 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:31.544146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:31.554408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41418-0080-4c50-b572-5cd444915b0d service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:31.610378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:31.611284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:31.611567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:31.611846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] PATH_INFO: `/v3/users/fb0876835ae24c4abeedf4b62a087a41` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:31.613022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0876835ae24c4abeedf4b62a087a41)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:31.614985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:31.660551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0876835ae24c4abeedf4b62a087a41'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.667582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0876835ae24c4abeedf4b62a087a41'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.683781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0876835ae24c4abeedf4b62a087a41'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.684701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0876835ae24c4abeedf4b62a087a41'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.685201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-33a39f3e-a1f9-49bc-8c3f-62c37bbb5e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0876835ae24c4abeedf4b62a087a41'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:31.697981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1020/2070] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:31 2022] DELETE /identity/v3/users/fb0876835ae24c4abeedf4b62a087a41 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:31.988308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:31.990105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:32.009949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb25272-ee2d-40a0-baa6-280b36347572 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.076329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.077205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.077474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.077772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] PATH_INFO: `/v3/projects/fba03f1e5d124d7eb931a8b312881bea` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.079198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba03f1e5d124d7eb931a8b312881bea)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.081693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.083171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:32.129112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba03f1e5d124d7eb931a8b312881bea', 'request_id': 'req-409e89a0-41a3-4f5f-80c6-905e5b62ab92'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:32.135040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba03f1e5d124d7eb931a8b312881bea', 'request_id': 'req-409e89a0-41a3-4f5f-80c6-905e5b62ab92'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:32.140379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba03f1e5d124d7eb931a8b312881bea', 'request_id': 'req-409e89a0-41a3-4f5f-80c6-905e5b62ab92'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:32.154222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] The token cache is being invalidate because project fba03f1e5d124d7eb931a8b312881bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:32.154514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:32.155170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-409e89a0-41a3-4f5f-80c6-905e5b62ab92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:32.156942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1051/2071] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:31 2022] DELETE /identity/v3/projects/fba03f1e5d124d7eb931a8b312881bea => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:32.161893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.162294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.162294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.182192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:32.183435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:32.230670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:32.244464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1021/2072] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:32.252316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.252609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.252932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.270823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:32.272036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:32.281497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1052/2073] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:32.286267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.296030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3721e2-8534-4d1a-9141-fd03f75a8011 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.301475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.302127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.302411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.302723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.303449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:32.303751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.305392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.317072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1022/2074] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:32.321246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.330506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf0a9ef-3dbc-43af-9b31-2fd4e46d1106 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.335198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.335844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.336130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.336407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.337108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.339046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.364472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1053/2075] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:32.368466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.378164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad90aa-711f-4ca5-b08d-5f39d4f42164 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.432320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.432913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.433194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.433469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.434142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.436069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.455990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1023/2076] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/users => generated 291 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:32.460059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.470047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43a95a-e1d1-4591-947d-ce5ac7c3680d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.475037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.475684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.475965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.476239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.476966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.478545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.489227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1054/2077] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:32.493028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.502739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d1ff2-6b4e-4597-a2a9-8ec224b72c54 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.507517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aca508-f3c4-4617-bb68-e9e5d810d986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.508162 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aca508-f3c4-4617-bb68-e9e5d810d986 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.508441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aca508-f3c4-4617-bb68-e9e5d810d986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.508714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aca508-f3c4-4617-bb68-e9e5d810d986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.509432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aca508-f3c4-4617-bb68-e9e5d810d986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.511062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aca508-f3c4-4617-bb68-e9e5d810d986 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.520831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1024/2078] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:32.524876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.535174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef85bb-4362-4bcd-a179-7538df1a2bf0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.539866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.540729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.541012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.541291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] PATH_INFO: `/v3/projects/1e088729d0e241568d8e3aa553ef6a29/users/802db3995d03409a853fe2c237b3ea97/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.556293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e088729d0e241568d8e3aa553ef6a29, user_id=802db3995d03409a853fe2c237b3ea97, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.558969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.571607 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1055/2079] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:32 2022] PUT /identity/v3/projects/1e088729d0e241568d8e3aa553ef6a29/users/802db3995d03409a853fe2c237b3ea97/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:32.582040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.582355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.582659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.609212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] MFA Rules not processed for user `802db3995d03409a853fe2c237b3ea97`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:32.610417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:32.674718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1025/2080] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:32.679019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.739339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a097d-7ab5-412a-aa89-7fe24e1dd0d3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.743793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.744437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.744719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.744991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.745686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.747602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.771968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1056/2081] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:32.776382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.787099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cbf426-1b3a-4b84-8456-de01a117dc3b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.832520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:32.845184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.845869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.846139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.846459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.847203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.849112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.869132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1026/2082] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:32 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:32.873155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.884692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4bc9-a3f3-4a6e-bf6a-b422a69abb8c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.889337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.889992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.890289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.890569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.891307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.892918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.898389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:32.906987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1057/2083] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:32.908594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.918383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf718b5-fa17-4132-bb40-6b4a9c9cb3e1 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:32.923001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:32.923869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:32.924152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:32.924429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] PATH_INFO: `/v3/projects/47b80ebc6e644776bfe012ed9d10a3c0/users/46cf59b9b0c849568c906bb436751377/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:32.939190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b80ebc6e644776bfe012ed9d10a3c0, user_id=46cf59b9b0c849568c906bb436751377, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:32.942472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:32.954513 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1027/2084] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:32 2022] PUT /identity/v3/projects/47b80ebc6e644776bfe012ed9d10a3c0/users/46cf59b9b0c849568c906bb436751377/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:32.958328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:32.968067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8b055-801c-4444-9593-00ffedeadc4a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.023630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c09fa-7e54-4c73-a6f6-e268d871c1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.024249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c09fa-7e54-4c73-a6f6-e268d871c1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.024497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c09fa-7e54-4c73-a6f6-e268d871c1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.024775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c09fa-7e54-4c73-a6f6-e268d871c1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.025500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c09fa-7e54-4c73-a6f6-e268d871c1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.027223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c09fa-7e54-4c73-a6f6-e268d871c1b6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.040933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1058/2085] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:33.042139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:33.053503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085fe77-7615-4380-9110-a17a447f1f04 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.058291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.059225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.059507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.059779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] PATH_INFO: `/v3/projects/47b80ebc6e644776bfe012ed9d10a3c0/users/46cf59b9b0c849568c906bb436751377/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.061738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] RBAC: Authorizing `identity:create_grant(project_id=47b80ebc6e644776bfe012ed9d10a3c0, user_id=46cf59b9b0c849568c906bb436751377, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.064420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.075481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1028/2086] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:33 2022] PUT /identity/v3/projects/47b80ebc6e644776bfe012ed9d10a3c0/users/46cf59b9b0c849568c906bb436751377/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:33.087219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.087598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.088051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.115636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] MFA Rules not processed for user `46cf59b9b0c849568c906bb436751377`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:33.116835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:33.183691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1059/2087] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:33.195595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:33.197170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:33.257814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3914b0-a79a-4572-a95b-c41161f5efae admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.261483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.262239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.262534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.262840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.272651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.274135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.283680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1029/2088] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:33.574507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:33.583964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19759bde-9334-45bc-a566-e324c284aaf5 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.588533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46cf59b9b0c849568c906bb436751377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b80ebc6e644776bfe012ed9d10a3c0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.589180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.589464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.589747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.590479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:33.590822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.592416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.604332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1060/2089] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:33.608282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:33.617763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619167c4-5978-4efb-ab72-2e2ca2bdc9a8 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.622300 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46cf59b9b0c849568c906bb436751377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b80ebc6e644776bfe012ed9d10a3c0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.622974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.623261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.623540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.624241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.626106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.651733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1030/2090] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:33 2022] POST /identity/v3/projects => generated 353 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:33.656029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:33.665792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fe02c-025c-4dc4-b73a-28a818e313f2 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.723091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46cf59b9b0c849568c906bb436751377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b80ebc6e644776bfe012ed9d10a3c0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.723830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.724120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.724401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.725125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:33.725435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.727062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.738611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1061/2091] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:33.772955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:33.783201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b941cd9-8c35-48fc-b456-a1b39ab6c666 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:33.788342 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46cf59b9b0c849568c906bb436751377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b80ebc6e644776bfe012ed9d10a3c0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:33.789118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:33.789415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:33.789697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] PATH_INFO: `/v3/projects/c6108a35412c4de4b4d4d3c907ccb76c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:33.790905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c6108a35412c4de4b4d4d3c907ccb76c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:33.792813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:33.837711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6108a35412c4de4b4d4d3c907ccb76c', 'request_id': 'req-3cdc0e30-500a-4243-acd1-f0f0d2be3739'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:33.845158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6108a35412c4de4b4d4d3c907ccb76c', 'request_id': 'req-3cdc0e30-500a-4243-acd1-f0f0d2be3739'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:33.851488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6108a35412c4de4b4d4d3c907ccb76c', 'request_id': 'req-3cdc0e30-500a-4243-acd1-f0f0d2be3739'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:33.863739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] The token cache is being invalidate because project c6108a35412c4de4b4d4d3c907ccb76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:33.864027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:33.864676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3cdc0e30-500a-4243-acd1-f0f0d2be3739 tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:33.866174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1031/2092] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:33 2022] DELETE /identity/v3/projects/c6108a35412c4de4b4d4d3c907ccb76c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:34.052101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:34.053990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:34.059285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:34.120623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbf546-9d5c-44a4-a16c-d8308c2cd42c tempest-VolumeQuotaClassesTest-1655113092 tempest-VolumeQuotaClassesTest-1655113092-project-admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:34.175738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:34.176614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:34.176900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:34.177179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] PATH_INFO: `/v3/users/802db3995d03409a853fe2c237b3ea97` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:34.178359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802db3995d03409a853fe2c237b3ea97)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:34.180356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:34.226338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802db3995d03409a853fe2c237b3ea97'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.233649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802db3995d03409a853fe2c237b3ea97'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.245492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802db3995d03409a853fe2c237b3ea97'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.246137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802db3995d03409a853fe2c237b3ea97'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.246783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802db3995d03409a853fe2c237b3ea97'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.259234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1062/2093] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:34 2022] DELETE /identity/v3/users/802db3995d03409a853fe2c237b3ea97 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:34.263479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:34.263905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:34.264113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:34.271545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:34.286595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:34.288010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:34.352916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1032/2094] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:34.362426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:34.363892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:34.395264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:34.421916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacb171-48bf-44ef-8384-157a170fe9b8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:34.425614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:34.426286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:34.426573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:34.426883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:34.436174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:34.437601 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:34.447471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1063/2095] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:34.711802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:34.713335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:34.727915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090241-3140-4d0d-9784-4d3071b39bb4 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:34.733900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:34.735079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:34.735464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:34.735741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] PATH_INFO: `/v3/projects/1e088729d0e241568d8e3aa553ef6a29` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:34.737085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e088729d0e241568d8e3aa553ef6a29)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:34.739140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:34.785953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e088729d0e241568d8e3aa553ef6a29', 'request_id': 'req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.791966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e088729d0e241568d8e3aa553ef6a29', 'request_id': 'req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.797430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e088729d0e241568d8e3aa553ef6a29', 'request_id': 'req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.809452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] The token cache is being invalidate because project 1e088729d0e241568d8e3aa553ef6a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:34.809677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.810281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.811860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1033/2096] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:34 2022] DELETE /identity/v3/projects/1e088729d0e241568d8e3aa553ef6a29 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:34.816470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:34.826120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ea8ad-b5fa-4141-99b6-bcf8103276ec service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:34.881272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:34.882133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:34.882415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:34.882719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] PATH_INFO: `/v3/users/46cf59b9b0c849568c906bb436751377` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:34.883901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cf59b9b0c849568c906bb436751377)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:34.885827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:34.929506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cf59b9b0c849568c906bb436751377'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.936251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cf59b9b0c849568c906bb436751377'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.948364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cf59b9b0c849568c906bb436751377'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.948993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cf59b9b0c849568c906bb436751377'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.949599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9a404a8b-79d3-4079-91a9-c6cf1fe6851b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cf59b9b0c849568c906bb436751377'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:34.961361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1064/2097] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:34 2022] DELETE /identity/v3/users/46cf59b9b0c849568c906bb436751377 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:35.270701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.274915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:35.293292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7bdc7-a5ed-4a12-a513-564dd6a978f5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.352692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.353578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.353865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.354143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] PATH_INFO: `/v3/projects/47b80ebc6e644776bfe012ed9d10a3c0` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.355397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] RBAC: Authorizing `identity:delete_project(project_id=47b80ebc6e644776bfe012ed9d10a3c0)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.357478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.391442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:35.405686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47b80ebc6e644776bfe012ed9d10a3c0', 'request_id': 'req-7e66a056-fced-409a-8976-3335c629b39e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:35.412238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47b80ebc6e644776bfe012ed9d10a3c0', 'request_id': 'req-7e66a056-fced-409a-8976-3335c629b39e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:35.417603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47b80ebc6e644776bfe012ed9d10a3c0', 'request_id': 'req-7e66a056-fced-409a-8976-3335c629b39e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:35.430794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] The token cache is being invalidate because project 47b80ebc6e644776bfe012ed9d10a3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:35.431083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:35.431700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7e66a056-fced-409a-8976-3335c629b39e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:35.433266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1034/2098] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:35 2022] DELETE /identity/v3/projects/47b80ebc6e644776bfe012ed9d10a3c0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:35.438182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.438663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.438772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.456750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:35.457942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:35.503264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:35.518816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1065/2099] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:35.526586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.526907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.527190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.545497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:35.546775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:35.555376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1035/2100] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:35.559354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.568845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91021688-d024-47ad-b235-b12f2afdd1d6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.573279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.573922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.574206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.574482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.575247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:35.575548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.577137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.588873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1066/2101] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:35.592837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.602634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7851d-f73b-450c-aed8-4436dc0be606 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.607311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.608579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.608870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.609150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.609845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.611776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.636346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1036/2102] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:35 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:35.640224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.651549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519b131-04f9-4c5c-817c-0e1e5332c6ea admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.707371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.708035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.708321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.708605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.709326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.711313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.729946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1067/2103] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:35 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:35.733953 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.744687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0c117-4f27-4a2f-ac88-11db7a3abc92 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.749406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.750066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.750356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.750670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.751407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.753699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.763954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1037/2104] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:35.767649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.778083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b3d8d-1159-4725-8437-0b18d42dfa34 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.782675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191d079-58a6-4289-ac6b-c2b273889abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.783331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191d079-58a6-4289-ac6b-c2b273889abe admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.783619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191d079-58a6-4289-ac6b-c2b273889abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.783900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191d079-58a6-4289-ac6b-c2b273889abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.784633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191d079-58a6-4289-ac6b-c2b273889abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.786233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191d079-58a6-4289-ac6b-c2b273889abe admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.796254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1068/2105] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:35.800566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:35.810394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1f14f-780e-41b5-96a5-25745b4adffd admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:35.815633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:35.816509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.816792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.817067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] PATH_INFO: `/v3/projects/75ed474307254556a9cb17160aaa6b11/users/fd0c1551ca534c6b896fa1cc6233859c/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.831247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ed474307254556a9cb17160aaa6b11, user_id=fd0c1551ca534c6b896fa1cc6233859c, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:35.833810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:35.846276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1038/2106] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:35 2022] PUT /identity/v3/projects/75ed474307254556a9cb17160aaa6b11/users/fd0c1551ca534c6b896fa1cc6233859c/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:35.856118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:35.856412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:35.856822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:35.883827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] MFA Rules not processed for user `fd0c1551ca534c6b896fa1cc6233859c`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:35.885034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:35.947993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1069/2107] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:35.952291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.010956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdada975-44fb-4e34-a3f3-8f9605796163 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.015484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.016128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.016415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.016690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.017371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.019304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.043692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1039/2108] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:35 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:36.047645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.058716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71a569-c9d7-4975-9349-82be592f1cf6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.102749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:36.117058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.117720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.118003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.118281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.119013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.120947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.143128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1070/2109] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:36 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:36.144201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.154985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76118a61-e2e3-4bd7-9993-0bdbe73e78b7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.160077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.160733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.161016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.161315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.162042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.163658 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.174220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1040/2110] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:36.179249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.195758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f07d4-cf41-4f43-90bb-866da18622f4 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.200895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.201910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.202199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.202504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] PATH_INFO: `/v3/projects/e1ae0d0aeb5442f5b97bf74a00d03ec2/users/3aa4d96f6ec64b82ae98c4d03e144e60/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.220474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ae0d0aeb5442f5b97bf74a00d03ec2, user_id=3aa4d96f6ec64b82ae98c4d03e144e60, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.224029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.238258 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1071/2111] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:36 2022] PUT /identity/v3/projects/e1ae0d0aeb5442f5b97bf74a00d03ec2/users/3aa4d96f6ec64b82ae98c4d03e144e60/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:36.250020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.260101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798c1a-aff2-44aa-9201-7c49c8193746 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.290555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:36.329848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61727cd0-1166-4b4d-87df-dd2543e921a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.330522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727cd0-1166-4b4d-87df-dd2543e921a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.331370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727cd0-1166-4b4d-87df-dd2543e921a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.331648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727cd0-1166-4b4d-87df-dd2543e921a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.332373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61727cd0-1166-4b4d-87df-dd2543e921a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.333972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61727cd0-1166-4b4d-87df-dd2543e921a3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.343651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1041/2112] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:36.348440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.358605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7c60a-c385-4038-b0b3-b1e0169683f0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.363326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.364200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.364487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.364764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] PATH_INFO: `/v3/projects/e1ae0d0aeb5442f5b97bf74a00d03ec2/users/3aa4d96f6ec64b82ae98c4d03e144e60/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.366802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ae0d0aeb5442f5b97bf74a00d03ec2, user_id=3aa4d96f6ec64b82ae98c4d03e144e60, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.369419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.386182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1072/2113] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:36 2022] PUT /identity/v3/projects/e1ae0d0aeb5442f5b97bf74a00d03ec2/users/3aa4d96f6ec64b82ae98c4d03e144e60/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:36.391476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.391786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.392054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.420228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] MFA Rules not processed for user `3aa4d96f6ec64b82ae98c4d03e144e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:36.421417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:36.487982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1042/2114] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:36.499016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.500734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:36.557986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9c62-5823-4f51-8767-d3840b1385a2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.561654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.562293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.562579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.562895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.571933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.573358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.583252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1073/2115] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:36.657071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:36.667281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca71c8-aab5-44c3-a9a7-ea826e32c3e9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:36.671009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1285bc-fdee-487c-98f3-99db0b066eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:36.671651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1285bc-fdee-487c-98f3-99db0b066eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:36.671937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1285bc-fdee-487c-98f3-99db0b066eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:36.672217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1285bc-fdee-487c-98f3-99db0b066eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:36.730350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1285bc-fdee-487c-98f3-99db0b066eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:36.731887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1285bc-fdee-487c-98f3-99db0b066eb6 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:36.741105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1043/2116] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:40.767830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:40.769254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:40.838686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b681d1c-a5fa-414f-8f01-02700d6b76de service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:40.843122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:40.843989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:40.844271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:40.844542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] PATH_INFO: `/v3/users/bd29b7d9022e4abd9dee0c596c310ee8` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:40.845738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd29b7d9022e4abd9dee0c596c310ee8)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:40.848321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:40.891752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:40.894847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd29b7d9022e4abd9dee0c596c310ee8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:40.904576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd29b7d9022e4abd9dee0c596c310ee8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:40.917837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd29b7d9022e4abd9dee0c596c310ee8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:40.918495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd29b7d9022e4abd9dee0c596c310ee8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:40.919162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd29b7d9022e4abd9dee0c596c310ee8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:40.931689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1074/2117] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:40 2022] DELETE /identity/v3/users/bd29b7d9022e4abd9dee0c596c310ee8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:40.936142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:40.936583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:40.936691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:40.954180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:40.955961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:41.001408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:41.017732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1044/2118] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:41.028025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:41.029455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:41.086424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3c9a0-1f7a-4ccc-90fd-1118ba0b6eee admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:41.090223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:41.090848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:41.091163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:41.091439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:41.101206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:41.102677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:41.112203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1075/2119] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:41.422304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:41.434942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:41.462176 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a9687-4eb3-4ee2-a463-9ee6c964fb28 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:41.469988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab1632-4d05-4276-a091-488469235579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:41.474649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab1632-4d05-4276-a091-488469235579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:41.474649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab1632-4d05-4276-a091-488469235579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:41.474649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab1632-4d05-4276-a091-488469235579 admin admin] PATH_INFO: `/v3/projects/651b2ebb2a41444788a568ee990fd2c2` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:41.474649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab1632-4d05-4276-a091-488469235579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=651b2ebb2a41444788a568ee990fd2c2)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:41.477295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab1632-4d05-4276-a091-488469235579 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:41.544542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651b2ebb2a41444788a568ee990fd2c2', 'request_id': 'req-beab1632-4d05-4276-a091-488469235579'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.552219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651b2ebb2a41444788a568ee990fd2c2', 'request_id': 'req-beab1632-4d05-4276-a091-488469235579'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.562003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651b2ebb2a41444788a568ee990fd2c2', 'request_id': 'req-beab1632-4d05-4276-a091-488469235579'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.577123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-beab1632-4d05-4276-a091-488469235579 admin admin] The token cache is being invalidate because project 651b2ebb2a41444788a568ee990fd2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:41.577518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.578242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.580294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1045/2120] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:41 2022] DELETE /identity/v3/projects/651b2ebb2a41444788a568ee990fd2c2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:41.584544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:41.595305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff568463-a579-43a0-ae6f-71fd757f3afb service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:41.679893 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:41.681481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:41.681835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:41.682147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] PATH_INFO: `/v3/users/bb1088aad76540678fe83c101ead44f3` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:41.683629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1088aad76540678fe83c101ead44f3)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:41.685977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:41.750034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1088aad76540678fe83c101ead44f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.762081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1088aad76540678fe83c101ead44f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.773588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:41.780479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1088aad76540678fe83c101ead44f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.781666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1088aad76540678fe83c101ead44f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.783052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1088aad76540678fe83c101ead44f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:41.801653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1076/2121] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:41 2022] DELETE /identity/v3/users/bb1088aad76540678fe83c101ead44f3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:42.152588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:42.154297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:42.174168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab1632-4d05-4276-a091-488469235579 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:42.294906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:42.295950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:42.296341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:42.296724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] PATH_INFO: `/v3/projects/cb35e650441e45c383f29929052a7a00` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:42.298481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb35e650441e45c383f29929052a7a00)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:42.302971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:42.403413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb35e650441e45c383f29929052a7a00', 'request_id': 'req-a55d0121-fbac-48f1-8fff-3178c9433372'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.413451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb35e650441e45c383f29929052a7a00', 'request_id': 'req-a55d0121-fbac-48f1-8fff-3178c9433372'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.420943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb35e650441e45c383f29929052a7a00', 'request_id': 'req-a55d0121-fbac-48f1-8fff-3178c9433372'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.438131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] The token cache is being invalidate because project cb35e650441e45c383f29929052a7a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:42.438763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.439575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.444891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1046/2122] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:42 2022] DELETE /identity/v3/projects/cb35e650441e45c383f29929052a7a00 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:42.445752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:42.448279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:42.472605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6d8bd-ab43-4095-931b-32de22874342 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:42.545386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:42.546369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:42.546860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:42.547241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] PATH_INFO: `/v3/users/4cb2230b1c2342b4be5cbd572cc2a244` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:42.548874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb2230b1c2342b4be5cbd572cc2a244)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:42.551387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:42.637123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb2230b1c2342b4be5cbd572cc2a244'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.656353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb2230b1c2342b4be5cbd572cc2a244'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.676190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb2230b1c2342b4be5cbd572cc2a244'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.691978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb2230b1c2342b4be5cbd572cc2a244'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.699840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2d227f4b-2ae1-4070-91f6-2821033fc87d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb2230b1c2342b4be5cbd572cc2a244'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:42.716681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1077/2123] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:42 2022] DELETE /identity/v3/users/4cb2230b1c2342b4be5cbd572cc2a244 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:43.009599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.011649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:43.034713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:43.038406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d0121-fbac-48f1-8fff-3178c9433372 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.128320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.128573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.128948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.130590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] PATH_INFO: `/v3/projects/bdc3fd92e43b40289bbb89a4be5b8247` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.130590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc3fd92e43b40289bbb89a4be5b8247)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.132763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.187806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc3fd92e43b40289bbb89a4be5b8247', 'request_id': 'req-b8eabd8a-3fcd-4e95-8be1-05319396cb42'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.195062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc3fd92e43b40289bbb89a4be5b8247', 'request_id': 'req-b8eabd8a-3fcd-4e95-8be1-05319396cb42'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.201458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc3fd92e43b40289bbb89a4be5b8247', 'request_id': 'req-b8eabd8a-3fcd-4e95-8be1-05319396cb42'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.219125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] The token cache is being invalidate because project bdc3fd92e43b40289bbb89a4be5b8247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:43.219492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.220195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b8eabd8a-3fcd-4e95-8be1-05319396cb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.222444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1047/2124] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:43 2022] DELETE /identity/v3/projects/bdc3fd92e43b40289bbb89a4be5b8247 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:43.228313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.229121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.229550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.248779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:43.249657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:43.321259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1078/2125] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:43.329490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.329809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.330089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.349615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:43.351186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:43.363725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1048/2126] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:43.368080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.379335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:43.393487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dab6b-5854-4f8c-86ea-a4d111dad95b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.398609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.399218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.399505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.399786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.400629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:43.400842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.402513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.418672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1079/2127] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:43.419566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.432839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186b02-0292-4f8f-98fd-f7c160e2c550 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.438765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.439473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.440386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.440729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.441519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.443556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.482668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1049/2128] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:43 2022] POST /identity/v3/projects => generated 397 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:06:43.485031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.494364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.498657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68384c-a6d6-494b-8f38-c2f9d247e1a1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.500500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:43.577574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.578680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.579149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.579636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.580562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.582889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.596891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ec80-26ff-4618-abdc-cb525872bcbc admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.603437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.604566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.604975 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.605318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] PATH_INFO: `/v3/users/3a2912f203e44d1ab0162fbc205db0b3` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.607960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2912f203e44d1ab0162fbc205db0b3)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.610536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1080/2129] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:43 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:43.610937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.615028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.619877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:43.627888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fba04-a9b3-47c8-a55a-47422c5c568c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.633617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.635085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.635894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.636166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.636980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.639229 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.654429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1081/2130] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:43.656231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.669952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2994507-842e-4e20-abe6-cfc42a3a29fc admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.676377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.677275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.677684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.678111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.679215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.681402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.691006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2912f203e44d1ab0162fbc205db0b3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.695376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1082/2131] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:43.699297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.703270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2912f203e44d1ab0162fbc205db0b3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.713830 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691a5fd-bc93-4886-b068-59bfe4401ba3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:43.719802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2912f203e44d1ab0162fbc205db0b3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.720571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2912f203e44d1ab0162fbc205db0b3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.721790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e8feb903-fd10-4855-8c3a-967d8af47c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2912f203e44d1ab0162fbc205db0b3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:43.722084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a9cb-5871-47dd-bb66-603278854ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:43.723447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a9cb-5871-47dd-bb66-603278854ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.723770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a9cb-5871-47dd-bb66-603278854ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.724562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a9cb-5871-47dd-bb66-603278854ead admin admin] PATH_INFO: `/v3/projects/50afbcabe0ee43a1a2a499b57bf0cde0/users/03dd0cd5f59b4bbe9773c78e975d3f44/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.746146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a9cb-5871-47dd-bb66-603278854ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=50afbcabe0ee43a1a2a499b57bf0cde0, user_id=03dd0cd5f59b4bbe9773c78e975d3f44, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:43.749738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a9cb-5871-47dd-bb66-603278854ead admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:43.767939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1050/2132] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:43 2022] DELETE /identity/v3/users/3a2912f203e44d1ab0162fbc205db0b3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:43.769574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.769838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.770116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.780538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1083/2133] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:43 2022] PUT /identity/v3/projects/50afbcabe0ee43a1a2a499b57bf0cde0/users/03dd0cd5f59b4bbe9773c78e975d3f44/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:43.811987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:43.813288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:43.859629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:43.859925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:43.860211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:43.889918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] MFA Rules not processed for user `03dd0cd5f59b4bbe9773c78e975d3f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:43.891513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:43.934732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1051/2134] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:43.942416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:43.944324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.001496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1084/2135] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:44.008622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:44.011150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.031231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d45644-96b3-4ea0-a487-01554867fe21 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:44.038638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:44.039801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:44.040235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:44.040826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] PATH_INFO: `/v3/users/fd0c1551ca534c6b896fa1cc6233859c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:44.042257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0c1551ca534c6b896fa1cc6233859c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:44.044962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:44.055486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839cd401-a438-4d86-b3f5-c90f3aa63f7b None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:44.059857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:44.060744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:44.061037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:44.061359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:44.064407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:44.076742 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:44.078270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:44.096000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1052/2136] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:44.097241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:44.098684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.121350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0c1551ca534c6b896fa1cc6233859c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:44.136941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:44.142491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0c1551ca534c6b896fa1cc6233859c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:44.179904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0c1551ca534c6b896fa1cc6233859c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:44.180643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0c1551ca534c6b896fa1cc6233859c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:44.182180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-36134720-776d-43ae-a7af-4f1c5e16094b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0c1551ca534c6b896fa1cc6233859c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:44.201184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65464c96-da0d-4df8-9b01-e7f6ed8fee4f service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:44.203109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1085/2137] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:44 2022] DELETE /identity/v3/users/fd0c1551ca534c6b896fa1cc6233859c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:44.207662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:44.208155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:44.208576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:44.257765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:44.262905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.292266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:44.293049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:44.293993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:44.293993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:44.296225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.393397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1086/2138] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:44.396064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:44.403967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:44.404697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:44.407188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.415988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1053/2139] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:44.477752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e29f2-dfef-4287-b183-ac54db898c3a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:44.482610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4876a8-0fbc-4ae8-be60-fd7a4aa3824a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:44.483342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4876a8-0fbc-4ae8-be60-fd7a4aa3824a service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:44.483572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4876a8-0fbc-4ae8-be60-fd7a4aa3824a service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:44.483981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4876a8-0fbc-4ae8-be60-fd7a4aa3824a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:44.499179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4876a8-0fbc-4ae8-be60-fd7a4aa3824a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:44.500968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4876a8-0fbc-4ae8-be60-fd7a4aa3824a service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:44.513205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1087/2140] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:06:44.839120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:44.840564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:44.867103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59493cfa-1f34-47f0-91fe-3e2a068ece2a service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:44.892474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:44.897747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:44.900173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:44.900677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] PATH_INFO: `/v3/projects/992945dc7186465d8969d3013feed074` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:44.907054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=992945dc7186465d8969d3013feed074)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:44.910175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.059746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '992945dc7186465d8969d3013feed074', 'request_id': 'req-4346fc96-8fcf-476d-9725-71513447def4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.069397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '992945dc7186465d8969d3013feed074', 'request_id': 'req-4346fc96-8fcf-476d-9725-71513447def4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.085564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '992945dc7186465d8969d3013feed074', 'request_id': 'req-4346fc96-8fcf-476d-9725-71513447def4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.113141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] The token cache is being invalidate because project 992945dc7186465d8969d3013feed074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:45.113141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.117201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4346fc96-8fcf-476d-9725-71513447def4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.122731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1054/2141] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:44 2022] DELETE /identity/v3/projects/992945dc7186465d8969d3013feed074 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:45.126189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.126482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.126806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.148622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:45.150267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.189113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:45.238907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.240113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.240540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.242678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1088/2142] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:45.264154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:45.266668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.285031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1055/2143] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:45.289532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.300558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdb1cb-e4a7-48f4-9793-10009e5966d7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.305763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.306440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.319512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.319966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.321264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:45.321772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.324182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.339302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1089/2144] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:45.343305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.352589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c2eb5-7d5c-44ed-a3fc-181cf0071935 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.357332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.357980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.358264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.358541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.359307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.361235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.371666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:45.388504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1056/2145] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:45 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:45.392643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.403021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072186e-1f53-4ee6-8329-68137f98e4e0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.467667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.468370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.468658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.468950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.469626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.471738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.492497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1090/2146] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:45 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:45.497073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.497574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.499503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.509772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888af61-01f8-48e7-9be0-95861758329d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.514847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a7c95-98bf-4eef-8587-25059395b37e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.515600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f7129-85ae-4ae6-9ffe-89c710aed7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.517113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f7129-85ae-4ae6-9ffe-89c710aed7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.517398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f7129-85ae-4ae6-9ffe-89c710aed7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.517710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f7129-85ae-4ae6-9ffe-89c710aed7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.518455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f7129-85ae-4ae6-9ffe-89c710aed7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.520378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f7129-85ae-4ae6-9ffe-89c710aed7e2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.520629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.521627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.522261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.522774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] PATH_INFO: `/v3/projects/75ed474307254556a9cb17160aaa6b11` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.524865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ed474307254556a9cb17160aaa6b11)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.527409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.536949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1091/2147] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:45.536949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdd44a-8f48-401f-9e58-3f193695db53 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.536949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdd44a-8f48-401f-9e58-3f193695db53 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.536949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdd44a-8f48-401f-9e58-3f193695db53 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.538565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1092/2148] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:06:45.539746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdd44a-8f48-401f-9e58-3f193695db53 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.554250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdd44a-8f48-401f-9e58-3f193695db53 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.563852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209bc9ac-ee65-46ad-8ca9-364dec751ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.567980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc9ac-ee65-46ad-8ca9-364dec751ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.568887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc9ac-ee65-46ad-8ca9-364dec751ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.569261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209bc9ac-ee65-46ad-8ca9-364dec751ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.570008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bc9ac-ee65-46ad-8ca9-364dec751ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.571890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209bc9ac-ee65-46ad-8ca9-364dec751ebf admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.583985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1093/2149] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:45.588908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.589195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.589637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.597514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ed474307254556a9cb17160aaa6b11', 'request_id': 'req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.606699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ed474307254556a9cb17160aaa6b11', 'request_id': 'req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.616966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:45.616966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.617453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ed474307254556a9cb17160aaa6b11', 'request_id': 'req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.635150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] The token cache is being invalidate because project 75ed474307254556a9cb17160aaa6b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:45.635698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.636425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.640705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1057/2150] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:45 2022] DELETE /identity/v3/projects/75ed474307254556a9cb17160aaa6b11 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:45.640705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.640705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.691138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1094/2151] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:45.692088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.693374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.712089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1614-1c84-4e8a-bab4-8779eac5b2b5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.717078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.717965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.718246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.718516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] PATH_INFO: `/v3/projects/c09f02b3463a47e68081b8b47cc86985/users/0c321d34fcce4054849926e924b3c0a8/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.726371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:45.734151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09f02b3463a47e68081b8b47cc86985, user_id=0c321d34fcce4054849926e924b3c0a8, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.737403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.750327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1058/2152] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:45 2022] PUT /identity/v3/projects/c09f02b3463a47e68081b8b47cc86985/users/0c321d34fcce4054849926e924b3c0a8/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:45.751082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.752659 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.763014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c0cfc-85fa-4c8f-bcc2-eb6536d20321 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.814063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:45.817695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb98b2-97b2-47bb-aa56-c4a3b03ff99b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:45.820570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.821620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.822072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.822412 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] PATH_INFO: `/v3/users/a28051d0f956461c8f3b39825e33af96` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.823662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a28051d0f956461c8f3b39825e33af96)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.823825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:45.824781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.825507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.825647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.826060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] PATH_INFO: `/v3/users/3aa4d96f6ec64b82ae98c4d03e144e60` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.827376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa4d96f6ec64b82ae98c4d03e144e60)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:45.829652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:45.873505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a28051d0f956461c8f3b39825e33af96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.878058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa4d96f6ec64b82ae98c4d03e144e60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.881531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a28051d0f956461c8f3b39825e33af96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.885453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa4d96f6ec64b82ae98c4d03e144e60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.894791 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a28051d0f956461c8f3b39825e33af96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.895491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a28051d0f956461c8f3b39825e33af96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.896118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a28051d0f956461c8f3b39825e33af96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.897970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa4d96f6ec64b82ae98c4d03e144e60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.898639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa4d96f6ec64b82ae98c4d03e144e60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.899283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-64185c39-edff-43fb-96f6-379470094d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa4d96f6ec64b82ae98c4d03e144e60'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:45.909712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1095/2153] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:45 2022] DELETE /identity/v3/users/a28051d0f956461c8f3b39825e33af96 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:45.910549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:45.911953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.914920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1059/2154] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:45 2022] DELETE /identity/v3/users/3aa4d96f6ec64b82ae98c4d03e144e60 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:45.914920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:45.914920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:45.914920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:45.932734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:45.934083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:45.974889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:45.998283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512b99d-3dfc-497e-820a-b9c84ff6af24 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.010766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1060/2155] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:46.024650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.027231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.085967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.086864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.087268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.087646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.090497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.115724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.121060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.141497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c32214-b769-46bc-a1e8-877e01ed3579 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.146054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.147246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.147534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.147812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.150464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1096/2156] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:46.163134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.164666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.177343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1061/2157] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:46.274176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.275342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.281225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.282814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.293610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995510-524b-4e4f-bc29-ad4918dca1b0 service placement] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.299836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e16d-e490-43e8-874a-6e7ad39d5614 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.305399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.306397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.306726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.307001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] PATH_INFO: `/v3/projects/e1ae0d0aeb5442f5b97bf74a00d03ec2` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.308209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ae0d0aeb5442f5b97bf74a00d03ec2)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.308833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.309834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.310134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.310449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.310449 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] PATH_INFO: `/v3/projects/9eb5da85f8a74b08aef7b37cc209ca92` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.311922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb5da85f8a74b08aef7b37cc209ca92)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.313943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.332230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:46.394658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb5da85f8a74b08aef7b37cc209ca92', 'request_id': 'req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.404527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb5da85f8a74b08aef7b37cc209ca92', 'request_id': 'req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.408943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ae0d0aeb5442f5b97bf74a00d03ec2', 'request_id': 'req-939fc506-fe99-42e1-bc0e-25c2a957b84c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.416816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb5da85f8a74b08aef7b37cc209ca92', 'request_id': 'req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.417669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ae0d0aeb5442f5b97bf74a00d03ec2', 'request_id': 'req-939fc506-fe99-42e1-bc0e-25c2a957b84c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.426872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ae0d0aeb5442f5b97bf74a00d03ec2', 'request_id': 'req-939fc506-fe99-42e1-bc0e-25c2a957b84c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.441738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] The token cache is being invalidate because project 9eb5da85f8a74b08aef7b37cc209ca92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:46.442184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.442989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] The token cache is being invalidate because project e1ae0d0aeb5442f5b97bf74a00d03ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:46.443217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-79c2e14a-9082-4bb2-96fd-bc8ab90e1a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.443335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.443956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-939fc506-fe99-42e1-bc0e-25c2a957b84c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:46.446304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1097/2158] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:46 2022] DELETE /identity/v3/projects/9eb5da85f8a74b08aef7b37cc209ca92 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:46.447527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1062/2159] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:46 2022] DELETE /identity/v3/projects/e1ae0d0aeb5442f5b97bf74a00d03ec2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:46.452082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f84bc-87c6-414e-9075-114b0d5cd032 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.452385 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f84bc-87c6-414e-9075-114b0d5cd032 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.452664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f84bc-87c6-414e-9075-114b0d5cd032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.455926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeba40e-d384-4201-bdde-a57de612dccb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.456243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeba40e-d384-4201-bdde-a57de612dccb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.456519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeba40e-d384-4201-bdde-a57de612dccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.476669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-966f84bc-87c6-414e-9075-114b0d5cd032 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:46.477820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ddeba40e-d384-4201-bdde-a57de612dccb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:46.478480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-966f84bc-87c6-414e-9075-114b0d5cd032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.479009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ddeba40e-d384-4201-bdde-a57de612dccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.546260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1098/2160] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:46.553659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.553949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.554231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.562922 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1063/2161] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:46.565033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.565676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.565931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.575703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:46.577208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.590754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1099/2162] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:46.593137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.593902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:46.597010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:46.605125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:46.612599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801b38-5cc0-4501-9a3f-32c1402d09c4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.613011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1064/2163] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:46.618484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.631970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68828d4-4635-44db-8c77-a42867b97ff9 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.636458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.637439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.637944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.638443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.639114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.639787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.640072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.640353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.640466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:46.640982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.641098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:46.641382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.643580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.644240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.660736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1065/2164] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:06:46.665619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.672491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1100/2165] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:46.676673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.684837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91572c5-007a-45ad-b08c-79323d217d83 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.686892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16792e0-2288-41c9-a135-a2ce717f4fec admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.690521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.691406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.691753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.692021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.692645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.692767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.693387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.693741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.694094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.694635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.694986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.697022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.731212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1101/2166] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:46.736243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.736926 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1066/2167] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/projects => generated 391 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:46.740870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.805271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dde7b5-82eb-45a7-83a3-69b28e84f3f1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.810969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.811653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.811783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139e9c5-abdb-4dd0-8769-d12cf5eeb85d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.811973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.812221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.812921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.815550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.817059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.817733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.818017 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.818295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.819023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.820954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.829500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:46.839633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1102/2168] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:46.844467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.853779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1067/2169] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:06:46.854961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.856135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e749b-177d-4446-8a38-3a780b0b5a54 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.862336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.863034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.863473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.863884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.864680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.865570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077db9-5a7a-4808-a941-70fed2b48d53 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.866296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.872409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.873078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.873487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.873931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.875057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.876280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.878870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1103/2170] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:46.882778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.889731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1068/2171] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:46.894217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.895179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0e5e0-5781-4ded-9c03-b4db5d7b38b5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.900814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.901488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.901825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.902266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.902808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.904418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.905959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66593-3d7a-4820-a3c3-ca4ccdf523fb admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.913088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.913553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.913831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.914108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.914882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.918802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1104/2172] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:46.918955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.919928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.932047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1069/2173] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:46.932331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58531122-f576-4c6b-82fa-78bf13c117b3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.935908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:46.937310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3fc7-89f5-4717-ae5f-0250d7f19181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.938473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3fc7-89f5-4717-ae5f-0250d7f19181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.939057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3fc7-89f5-4717-ae5f-0250d7f19181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.939456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3fc7-89f5-4717-ae5f-0250d7f19181 admin admin] PATH_INFO: `/v3/projects/00f6b6012ccf4df487c1749fb8eb7c21/users/cbd63185e0444c3e8969dd63e57ba0ef/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.948117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c37f47-7c81-45e6-8085-dafde7d9fe71 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:46.953992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7fc39-6eb3-4068-b102-66051e28c5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:46.955019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7fc39-6eb3-4068-b102-66051e28c5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:46.955257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7fc39-6eb3-4068-b102-66051e28c5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:46.955527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7fc39-6eb3-4068-b102-66051e28c5ac admin admin] PATH_INFO: `/v3/projects/c1d4119a1d0b474fb568818f1aaa933b/users/dc40c07fceab4a83ba9324da868034aa/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:46.962333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3fc7-89f5-4717-ae5f-0250d7f19181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f6b6012ccf4df487c1749fb8eb7c21, user_id=cbd63185e0444c3e8969dd63e57ba0ef, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.967682 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3fc7-89f5-4717-ae5f-0250d7f19181 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.982006 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7fc39-6eb3-4068-b102-66051e28c5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d4119a1d0b474fb568818f1aaa933b, user_id=dc40c07fceab4a83ba9324da868034aa, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:46.985098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7fc39-6eb3-4068-b102-66051e28c5ac admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:46.989176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1105/2174] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:46 2022] PUT /identity/v3/projects/00f6b6012ccf4df487c1749fb8eb7c21/users/cbd63185e0444c3e8969dd63e57ba0ef/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:47.000058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.000479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.001006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.018370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1070/2175] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:46 2022] PUT /identity/v3/projects/c1d4119a1d0b474fb568818f1aaa933b/users/dc40c07fceab4a83ba9324da868034aa/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:47.024866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.025948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.026051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.042268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] MFA Rules not processed for user `cbd63185e0444c3e8969dd63e57ba0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:47.043772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:47.055712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:47.057761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:47.140862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1071/2176] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:47.151678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.155436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:47.158298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1106/2177] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:47.163177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.169261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:47.181736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893778c-df92-4f9c-b3ba-441b57002565 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.187287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.187982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.188264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.188545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.189233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.192949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.238881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1107/2178] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:47 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:47.240387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.255841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447afbf-f580-460b-80bb-a00f27e3107f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.317863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.318754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.319512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.319512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.331454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59d68b-7ee8-4e8a-ab7c-164e648c0040 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.334680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:47.337081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.337959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.338388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.338768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.339613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.342189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.373271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1108/2179] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:47 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:47.374778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.391377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf93534-bee3-4a40-9388-020c29af3210 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.395126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.397189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.397580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.398390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.398707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.399076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.400815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.401526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.410606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1072/2180] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:47.427465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1109/2181] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:47.429207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.442836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8a44f-7a08-41a2-881e-9a92ee884da5 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.449578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.450696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.451105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.451454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] PATH_INFO: `/v3/projects/be515fa978754be88a5139aebe19f090/users/0fcf66f8dac044a39b15cbd497548a6a/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.479867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=be515fa978754be88a5139aebe19f090, user_id=0fcf66f8dac044a39b15cbd497548a6a, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.483784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.505716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1073/2182] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:47 2022] PUT /identity/v3/projects/be515fa978754be88a5139aebe19f090/users/0fcf66f8dac044a39b15cbd497548a6a/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:47.510994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.527402 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8706e-8c04-4a9c-85c5-3701bbcf15fb admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.617512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63314816-e55d-48e8-ae97-111f705b6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.618222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314816-e55d-48e8-ae97-111f705b6bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.618506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314816-e55d-48e8-ae97-111f705b6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.618828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314816-e55d-48e8-ae97-111f705b6bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.619677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63314816-e55d-48e8-ae97-111f705b6bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.621763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63314816-e55d-48e8-ae97-111f705b6bc4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.634376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1110/2183] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:47.638195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.662796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359ef46-af2b-470f-bc24-6e153c200dce admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:47.685211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:47.689038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.690314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.691790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] PATH_INFO: `/v3/projects/be515fa978754be88a5139aebe19f090/users/0fcf66f8dac044a39b15cbd497548a6a/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.697253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be515fa978754be88a5139aebe19f090, user_id=0fcf66f8dac044a39b15cbd497548a6a, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:47.702188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:47.729120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1074/2184] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:47 2022] PUT /identity/v3/projects/be515fa978754be88a5139aebe19f090/users/0fcf66f8dac044a39b15cbd497548a6a/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:47.730396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:47.731179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:47.731694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:47.767273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] MFA Rules not processed for user `0fcf66f8dac044a39b15cbd497548a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:47.769584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:47.827715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:47.884968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1111/2185] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:47.898801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:47.900887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:48.001641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5469580-264d-4dca-8605-6929490eff55 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:48.005870 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bef44a-b080-4146-9408-83e7c2a57e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:48.006526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef44a-b080-4146-9408-83e7c2a57e8c service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:48.006843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef44a-b080-4146-9408-83e7c2a57e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:48.007123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bef44a-b080-4146-9408-83e7c2a57e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:48.023735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef44a-b080-4146-9408-83e7c2a57e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:48.025335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bef44a-b080-4146-9408-83e7c2a57e8c service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:48.047474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1075/2186] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:49.067114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:49.194226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0fbb86-716d-48a6-ab6c-b1e80c12a042 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:49.203262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:49.204921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:49.205439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:49.206042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] PATH_INFO: `/v3/users/cbd63185e0444c3e8969dd63e57ba0ef` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:49.208300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd63185e0444c3e8969dd63e57ba0ef)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:49.215929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:49.289809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd63185e0444c3e8969dd63e57ba0ef'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:49.300020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd63185e0444c3e8969dd63e57ba0ef'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:49.317289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd63185e0444c3e8969dd63e57ba0ef'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:49.318413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd63185e0444c3e8969dd63e57ba0ef'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:49.319276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd63185e0444c3e8969dd63e57ba0ef'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:49.338411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1112/2187] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:49 2022] DELETE /identity/v3/users/cbd63185e0444c3e8969dd63e57ba0ef => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:49.342672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:49.344982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:49.347271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:49.369410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:49.371749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:49.460496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1076/2188] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:49.504940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:49.507073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:49.665713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb611-3dea-4da6-98b4-f3d442ab9b9c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:49.671615 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:49.672529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:49.672992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:49.673428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:49.687788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:49.690652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:49.707670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1113/2189] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:50.543175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:50.544662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:50.562987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f15eda-10fc-4f84-bf8c-a8dbbf37ac7f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:50.576042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:50.576923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:50.577208 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:50.577482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] PATH_INFO: `/v3/projects/00f6b6012ccf4df487c1749fb8eb7c21` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:50.578846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f6b6012ccf4df487c1749fb8eb7c21)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:50.581902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:50.597860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:50.668999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f6b6012ccf4df487c1749fb8eb7c21', 'request_id': 'req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:50.682115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f6b6012ccf4df487c1749fb8eb7c21', 'request_id': 'req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:50.691631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f6b6012ccf4df487c1749fb8eb7c21', 'request_id': 'req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:50.714396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] The token cache is being invalidate because project 00f6b6012ccf4df487c1749fb8eb7c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:50.714989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:50.715688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6c3aa52c-c1b7-4ce2-bda9-abfea5b07cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:50.718878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1077/2190] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:50 2022] DELETE /identity/v3/projects/00f6b6012ccf4df487c1749fb8eb7c21 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:50.722217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:50.728668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:50.741052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831307a-1050-4b0f-9ef9-146586c32933 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:50.898155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:50.899676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:50.899915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:50.900204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] PATH_INFO: `/v3/users/0fcf66f8dac044a39b15cbd497548a6a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:50.901477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcf66f8dac044a39b15cbd497548a6a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:50.908884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:50.995531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcf66f8dac044a39b15cbd497548a6a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:51.019734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcf66f8dac044a39b15cbd497548a6a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:51.043304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcf66f8dac044a39b15cbd497548a6a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:51.044132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcf66f8dac044a39b15cbd497548a6a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:51.045179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcf66f8dac044a39b15cbd497548a6a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:51.078142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1114/2191] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:50 2022] DELETE /identity/v3/users/0fcf66f8dac044a39b15cbd497548a6a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:51.924920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:51.925445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:51.925556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:51.972374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] MFA Rules not processed for user `0c321d34fcce4054849926e924b3c0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:51.975404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:51.982194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:51.984274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:52.134997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725f7e9-41c2-4cea-aa92-8d37c6320b07 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.137530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1078/2192] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:52.138533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.141369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:52.149416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.150316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.150585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.151674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] PATH_INFO: `/v3/projects/be515fa978754be88a5139aebe19f090` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.157818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=be515fa978754be88a5139aebe19f090)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.162506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.177029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2dea5-e563-4609-9725-726d0c3d5f03 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.188538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.191433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.192102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.192593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.193598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.195835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.209209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:52.244751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1079/2193] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:52 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:52.249379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.263832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be515fa978754be88a5139aebe19f090', 'request_id': 'req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:52.272224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be515fa978754be88a5139aebe19f090', 'request_id': 'req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:52.279156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be515fa978754be88a5139aebe19f090', 'request_id': 'req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:52.295904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] The token cache is being invalidate because project be515fa978754be88a5139aebe19f090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:52.296553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:52.297471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-15ee6b76-fb5c-4047-b473-2f6e4f4fde0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:52.301139 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1115/2194] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:51 2022] DELETE /identity/v3/projects/be515fa978754be88a5139aebe19f090 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:52.307088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8b83-4d0e-4306-82ba-781f7a13ef2a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.307710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8b83-4d0e-4306-82ba-781f7a13ef2a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.308354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a8b83-4d0e-4306-82ba-781f7a13ef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.342879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c89b-1693-481f-9237-424eec45e6ef admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.350358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.351099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.351363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.351646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.352356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.355445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.358333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-719a8b83-4d0e-4306-82ba-781f7a13ef2a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:52.359623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-719a8b83-4d0e-4306-82ba-781f7a13ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:52.388232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1116/2195] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:52.390055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.391227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.391483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.394815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:52.432620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1080/2196] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:52 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:52.438442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.443131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:52.452812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:52.456560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:52.470242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9b13f-d2dc-4295-a90b-220f31367efd admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.478292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1117/2197] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:52.480066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.489135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.490392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.490876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.492405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.493322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.496714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.501057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b39b90-865a-4e5f-8827-e475e8ae4ed3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.512570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.513503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.529084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.540510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1081/2198] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:52.540721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.540721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:52.540721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.541146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.543381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.573276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ef893-406a-4041-83ef-7cc35678974d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.583194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1118/2199] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:52.588113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.602810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d221618-584a-42d4-9215-6e46d09024ec admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.608702 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.610217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.611126 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.612251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.613396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.615587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.616103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.619128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.619419 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.619697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] PATH_INFO: `/v3/projects/efdf91ab1557410db294ec94a183842f/users/33bd44d3ce8e479eaaae9c1145042ff8/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.668981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1119/2200] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:52.674370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.706205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=efdf91ab1557410db294ec94a183842f, user_id=33bd44d3ce8e479eaaae9c1145042ff8, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.719338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.773976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1082/2201] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:52 2022] PUT /identity/v3/projects/efdf91ab1557410db294ec94a183842f/users/33bd44d3ce8e479eaaae9c1145042ff8/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:52.786282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:52.824540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33ce5a-0bc2-464b-a866-3da3fd1e4948 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:52.934503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:52.935225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:52.935566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:52.935848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:52.936556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:52.938502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:52.963018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:52.974374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1120/2202] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:52 2022] POST /identity/v3/users => generated 292 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:52.977088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:53.007277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32af9c-14e3-4ed3-8d0b-10dd04163bae admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:53.013157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667a818-c9f9-4066-a5ea-14c88acd1b6d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:53.023765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:53.025834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.026117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.026394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.027184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:53.031015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:53.031930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.032062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:53.032495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.033581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.034497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:53.036330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:53.054408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1083/2203] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:53.058459 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:53.064784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1121/2204] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:53.069249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:53.107271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a3e24-a899-4353-81ad-ec75941438d6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:53.110233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26c2c7-d084-47c8-8b52-18f979302695 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:53.121275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:53.122717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.123731 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.124068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.125168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:53.128753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:53.133057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9789c-b0a0-4218-9e1b-4ed8847f6b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:53.134133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9789c-b0a0-4218-9e1b-4ed8847f6b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.134423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9789c-b0a0-4218-9e1b-4ed8847f6b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.136795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9789c-b0a0-4218-9e1b-4ed8847f6b64 admin admin] PATH_INFO: `/v3/projects/efdf91ab1557410db294ec94a183842f/users/33bd44d3ce8e479eaaae9c1145042ff8/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.145845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9789c-b0a0-4218-9e1b-4ed8847f6b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efdf91ab1557410db294ec94a183842f, user_id=33bd44d3ce8e479eaaae9c1145042ff8, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:53.164401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9789c-b0a0-4218-9e1b-4ed8847f6b64 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:53.183604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1122/2205] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:53.194780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1084/2206] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:53 2022] PUT /identity/v3/projects/efdf91ab1557410db294ec94a183842f/users/33bd44d3ce8e479eaaae9c1145042ff8/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:53.198849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:53.217068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c1fb-e2b7-411e-a886-5444c8f936df admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:53.339737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:53.343844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.344363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.344645 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] PATH_INFO: `/v3/projects/030adbefce3a4835853092b4561fd41f/users/fe3d557cc7fd4fefa20bd0309e24924f/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.380322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030adbefce3a4835853092b4561fd41f, user_id=fe3d557cc7fd4fefa20bd0309e24924f, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:53.389364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:53.420532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1123/2207] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:53 2022] PUT /identity/v3/projects/030adbefce3a4835853092b4561fd41f/users/fe3d557cc7fd4fefa20bd0309e24924f/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:53.432191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaf3ff-a9f6-4755-898a-c9ade1013535 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.432572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaf3ff-a9f6-4755-898a-c9ade1013535 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.433153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbaf3ff-a9f6-4755-898a-c9ade1013535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.529853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8bbaf3ff-a9f6-4755-898a-c9ade1013535 None None] MFA Rules not processed for user `fe3d557cc7fd4fefa20bd0309e24924f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:53.547310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8bbaf3ff-a9f6-4755-898a-c9ade1013535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:53.770168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1085/2208] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:53.776570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:53.820486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd1582-f35b-4c39-bdac-e0a96e3c48a6 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.820775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd1582-f35b-4c39-bdac-e0a96e3c48a6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.821047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd1582-f35b-4c39-bdac-e0a96e3c48a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.874085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-dddd1582-f35b-4c39-bdac-e0a96e3c48a6 None None] MFA Rules not processed for user `dc40c07fceab4a83ba9324da868034aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:53.875890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dddd1582-f35b-4c39-bdac-e0a96e3c48a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:53.910260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:53.957403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c132-e1d5-4895-80ba-2423fba19820 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:53.982590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:53.995880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:53.996219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:53.996773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:53.997510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.004759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.085335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1086/2209] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:54.097032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.097373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.097649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.098765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1124/2210] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:53 2022] POST /identity/v3/projects => generated 383 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:54.100218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.145351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] MFA Rules not processed for user `dc40c07fceab4a83ba9324da868034aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:54.147380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:54.207252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:54.223241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1087/2211] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:54.235022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692141f1-90ec-45b2-9966-5e8feb928bda admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.236886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.239384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:54.241340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.242255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.242604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.243001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.243779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.246336 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.287869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1125/2212] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:54 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:54.288842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.304268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b9816-9e45-4e8c-8cf0-ecccbe0b16b8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.315415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.316751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.317145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.317526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.318467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.320720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.343388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1126/2213] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:54.344429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.356244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6b4c4-4abf-4820-82b9-72b9e979a2eb admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.365396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.367037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.367324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.367606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] PATH_INFO: `/v3/projects/b5098f58e64f430191b20be8788e392d/users/3ed007a9ebe64c5c825b8c65ff38bb04/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.391392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5098f58e64f430191b20be8788e392d, user_id=3ed007a9ebe64c5c825b8c65ff38bb04, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.395949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.399261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91ef1-98f3-4e93-8ef0-9921a6a87d7f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.406200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.406914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.407332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.407635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.419404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.421200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.428214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1127/2214] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:54 2022] PUT /identity/v3/projects/b5098f58e64f430191b20be8788e392d/users/3ed007a9ebe64c5c825b8c65ff38bb04/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:54.429133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.442299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1088/2215] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:54.573966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190aab0-7a17-4030-a9c7-0ad551946363 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.580898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd56d-b148-4c1d-8e6a-1c94c7ca898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.581616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd56d-b148-4c1d-8e6a-1c94c7ca898c admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.582575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd56d-b148-4c1d-8e6a-1c94c7ca898c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.582922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd56d-b148-4c1d-8e6a-1c94c7ca898c admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.583728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd56d-b148-4c1d-8e6a-1c94c7ca898c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.586144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd56d-b148-4c1d-8e6a-1c94c7ca898c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.600628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1128/2216] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:54.604372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.619807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d97e6a-d569-4f03-94b6-1fb4acb4be1a service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.625182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.626136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.626472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.627376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] PATH_INFO: `/v3/projects/b5098f58e64f430191b20be8788e392d/users/3ed007a9ebe64c5c825b8c65ff38bb04/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.629605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5098f58e64f430191b20be8788e392d, user_id=3ed007a9ebe64c5c825b8c65ff38bb04, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.633046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:54.651038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1089/2217] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:54 2022] PUT /identity/v3/projects/b5098f58e64f430191b20be8788e392d/users/3ed007a9ebe64c5c825b8c65ff38bb04/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:06:54.661670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.661670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.662858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.708145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] MFA Rules not processed for user `3ed007a9ebe64c5c825b8c65ff38bb04`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:54.709468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:54.807289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1129/2218] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:54.815437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:54.817755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:54.916458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:54.939928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2dd10b-05fd-4836-ba45-67598afbc263 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:54.947339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:54.948306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:54.948727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:54.949020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:54.968096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:54.973593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:55.002340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1090/2219] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:55.088051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:55.103251 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cdc538-ef67-4177-9e33-c07a6e17da8d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:55.108415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f031b2-47cd-4b14-b3d8-7d8e988e930b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:55.109127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f031b2-47cd-4b14-b3d8-7d8e988e930b service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:55.109479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f031b2-47cd-4b14-b3d8-7d8e988e930b service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:55.109815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f031b2-47cd-4b14-b3d8-7d8e988e930b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:55.154654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:55.212855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f031b2-47cd-4b14-b3d8-7d8e988e930b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:55.214704 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f031b2-47cd-4b14-b3d8-7d8e988e930b service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:55.227453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1130/2220] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:55.607624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:55.695894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21bfc6-be14-4106-9caf-85a548692984 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:55.697205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21bfc6-be14-4106-9caf-85a548692984 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:55.697711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21bfc6-be14-4106-9caf-85a548692984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:55.734708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1c21bfc6-be14-4106-9caf-85a548692984 None None] MFA Rules not processed for user `dc40c07fceab4a83ba9324da868034aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:55.736289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1c21bfc6-be14-4106-9caf-85a548692984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:55.742474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f6ded-5ca9-416f-bbd7-7cbe3580d92b service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:55.756263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:55.757344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:55.757758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:55.758056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] PATH_INFO: `/v3/users/fe3d557cc7fd4fefa20bd0309e24924f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:55.759397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3d557cc7fd4fefa20bd0309e24924f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:55.765877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:55.847364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1131/2221] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:55.911667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3d557cc7fd4fefa20bd0309e24924f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:55.930698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3d557cc7fd4fefa20bd0309e24924f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:55.974881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3d557cc7fd4fefa20bd0309e24924f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:55.978275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3d557cc7fd4fefa20bd0309e24924f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:55.979075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3d557cc7fd4fefa20bd0309e24924f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:55.998850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1091/2222] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:55 2022] DELETE /identity/v3/users/fe3d557cc7fd4fefa20bd0309e24924f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:56.004991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:56.006014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:56.006910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:56.075860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:56.077205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:56.265745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1132/2223] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:56.295358 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:56.296836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:56.443661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc4e4e-6e07-49be-8551-b5e87566fa4d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:56.449204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:56.449879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:56.450164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:56.450444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:56.460934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:56.463121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:56.476512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1092/2224] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:06:56.793334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:56.794986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:56.818476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa854fe7-ef45-4b43-bcda-6f6943658668 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:56.835501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:56.836382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:56.836629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:56.836914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] PATH_INFO: `/v3/projects/030adbefce3a4835853092b4561fd41f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:56.838263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030adbefce3a4835853092b4561fd41f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:56.841557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:56.917869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030adbefce3a4835853092b4561fd41f', 'request_id': 'req-7e9b9d9d-3231-400e-9991-52aad6f11871'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:56.927078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030adbefce3a4835853092b4561fd41f', 'request_id': 'req-7e9b9d9d-3231-400e-9991-52aad6f11871'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:56.934133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:56.936574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030adbefce3a4835853092b4561fd41f', 'request_id': 'req-7e9b9d9d-3231-400e-9991-52aad6f11871'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:56.959000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] The token cache is being invalidate because project 030adbefce3a4835853092b4561fd41f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:56.959360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:56.960040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:56.965006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1133/2225] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:56 2022] DELETE /identity/v3/projects/030adbefce3a4835853092b4561fd41f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:56.966255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:56.977791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855f9a-c817-466f-8637-9bcb3367c741 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:56.981962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:57.114949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:57.115855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:57.116233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:57.116756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] PATH_INFO: `/v3/users/3ed007a9ebe64c5c825b8c65ff38bb04` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:57.125912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed007a9ebe64c5c825b8c65ff38bb04)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:57.128187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:57.249793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed007a9ebe64c5c825b8c65ff38bb04'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:57.273855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed007a9ebe64c5c825b8c65ff38bb04'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:57.317602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed007a9ebe64c5c825b8c65ff38bb04'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:57.319856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed007a9ebe64c5c825b8c65ff38bb04'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:57.321341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-80789432-5201-4da5-afbf-8f417353bd5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed007a9ebe64c5c825b8c65ff38bb04'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:57.348713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1093/2226] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:06:56 2022] DELETE /identity/v3/users/3ed007a9ebe64c5c825b8c65ff38bb04 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:57.999681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:58.001189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:58.031345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b9d9d-3231-400e-9991-52aad6f11871 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:58.190892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:58.192257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:58.192807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:58.193318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] PATH_INFO: `/v3/projects/b5098f58e64f430191b20be8788e392d` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:58.195146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5098f58e64f430191b20be8788e392d)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:58.197460 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:58.298229 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5098f58e64f430191b20be8788e392d', 'request_id': 'req-929a1ccf-c1a5-455b-9432-7259d02be297'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:58.307429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5098f58e64f430191b20be8788e392d', 'request_id': 'req-929a1ccf-c1a5-455b-9432-7259d02be297'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:58.327617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5098f58e64f430191b20be8788e392d', 'request_id': 'req-929a1ccf-c1a5-455b-9432-7259d02be297'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:58.360858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] The token cache is being invalidate because project b5098f58e64f430191b20be8788e392d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:06:58.361228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:58.362480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-929a1ccf-c1a5-455b-9432-7259d02be297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:06:58.364940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1134/2227] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:06:57 2022] DELETE /identity/v3/projects/b5098f58e64f430191b20be8788e392d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:58.370109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:58.371112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:58.371596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:58.395260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:58.397402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:58.527028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1094/2228] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:58.534883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:58.535176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:58.535458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:58.593242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:58.599249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:58.614566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1135/2229] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:06:58.621624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:58.694564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dac458-f8c0-4147-a664-be25fd9d098a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:58.703893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:58.707180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:58.709099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:58.711098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:58.715204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:06:58.718589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:58.723213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:58.742519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1095/2230] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:06:58.747155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:58.760188 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7c209-d40e-4f01-a7f2-eeb105012ed6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:58.804849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:58.806106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:58.806765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:58.807282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:58.808333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:58.810844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:58.907398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1136/2231] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:06:58 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:58.909036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:58.920755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604794bd-3e88-4ef2-be08-376ed4f823e0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:58.953695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:06:59.076371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:59.078800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:59.079511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:59.080482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:59.083270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:59.088192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:59.125055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1096/2232] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:06:58 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:06:59.128965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:59.161677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bd9a0-3fb5-4de6-8176-109b7274c2ef admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:59.167643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:59.168708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:59.169161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:59.170142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:59.172997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:59.175826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:59.194477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1137/2233] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:06:59.198670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:59.223787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c49d-df3f-47fb-9bd8-6aee4096d680 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:59.239083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9c649-721c-4486-a558-19312a096b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:59.239776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c649-721c-4486-a558-19312a096b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:59.240062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c649-721c-4486-a558-19312a096b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:59.242050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c649-721c-4486-a558-19312a096b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:59.242863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9c649-721c-4486-a558-19312a096b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:59.251906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9c649-721c-4486-a558-19312a096b3f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:59.328282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1097/2234] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:06:59.329499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:59.355719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48360d0a-f9e9-49d6-ab1f-ea8f746aef6c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:06:59.377312 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:06:59.378331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:59.378711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:59.379055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] PATH_INFO: `/v3/projects/2332a661874a4a2faebb32a154be663c/users/c9074c8e166b4cd2a3cff41f0ccca273/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:59.452127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2332a661874a4a2faebb32a154be663c, user_id=c9074c8e166b4cd2a3cff41f0ccca273, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:06:59.457145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:06:59.510939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1138/2235] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:06:59 2022] PUT /identity/v3/projects/2332a661874a4a2faebb32a154be663c/users/c9074c8e166b4cd2a3cff41f0ccca273/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:06:59.516322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae99e3-ad90-4577-983d-09306b09d324 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:06:59.517245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae99e3-ad90-4577-983d-09306b09d324 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:06:59.517641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae99e3-ad90-4577-983d-09306b09d324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:06:59.630393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-faae99e3-ad90-4577-983d-09306b09d324 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:06:59.635843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-faae99e3-ad90-4577-983d-09306b09d324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:06:59.838034 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1098/2236] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:06:59.983922 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:06:59.989170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:00.083113 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:00.331838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c14181-d906-488c-b5b7-06946d9b253e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:00.348699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:00.349585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:00.351567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:00.353539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:00.394638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:00.397362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:00.463845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1139/2237] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:00.823780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc953fb-db57-4db3-8145-877072802ca7 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:00.825154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc953fb-db57-4db3-8145-877072802ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:00.825447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc953fb-db57-4db3-8145-877072802ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:00.904304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ebc953fb-db57-4db3-8145-877072802ca7 None None] MFA Rules not processed for user `33bd44d3ce8e479eaaae9c1145042ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:00.905607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ebc953fb-db57-4db3-8145-877072802ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:01.033256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1099/2238] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:01.047873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:01.050345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:01.152839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b69b2-e58f-4ac4-8512-dbf6295e4f04 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:01.161103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:01.161569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:01.162035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:01.163117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:01.165701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:01.260285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:01.261746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:01.301445 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1140/2239] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:01.464231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:01.464523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:01.464811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:01.526803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:01.528291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:01.576771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:01.669454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1100/2240] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:01.749992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:01.752299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:01.890491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e1893-81c8-4178-9b15-032eda42a1c1 service glance] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:01.908141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3979ebbe-7837-4378-b57d-26810071d01a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:01.908832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ebbe-7837-4378-b57d-26810071d01a service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:01.921653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ebbe-7837-4378-b57d-26810071d01a service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:01.924552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3979ebbe-7837-4378-b57d-26810071d01a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:01.947203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979ebbe-7837-4378-b57d-26810071d01a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:01.955038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3979ebbe-7837-4378-b57d-26810071d01a service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:01.997676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1141/2241] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:07:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:07:02.148008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:02.150321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:02.174170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb10a7-2503-45fc-ab61-1e1c77197061 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:02.191268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:02.192531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:02.196592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:02.197080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] PATH_INFO: `/v3/users/03dd0cd5f59b4bbe9773c78e975d3f44` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:02.199370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03dd0cd5f59b4bbe9773c78e975d3f44)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:02.211173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:02.319116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03dd0cd5f59b4bbe9773c78e975d3f44'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:02.335549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03dd0cd5f59b4bbe9773c78e975d3f44'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:02.367197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03dd0cd5f59b4bbe9773c78e975d3f44'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:02.367968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03dd0cd5f59b4bbe9773c78e975d3f44'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:02.368610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03dd0cd5f59b4bbe9773c78e975d3f44'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:02.394927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1101/2242] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:07:02 2022] DELETE /identity/v3/users/03dd0cd5f59b4bbe9773c78e975d3f44 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:07:02.397125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:02.397593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:02.398045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:02.408684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:02.425487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:02.427119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:02.574864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1142/2243] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:02.619082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:02.620823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:02.740505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a88a65-62ee-4f7f-9009-4d2912d0dc23 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:02.755453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc90dea-51db-49da-867f-421b59df4606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:02.756149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc90dea-51db-49da-867f-421b59df4606 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:02.756440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc90dea-51db-49da-867f-421b59df4606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:02.756736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc90dea-51db-49da-867f-421b59df4606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:02.769597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc90dea-51db-49da-867f-421b59df4606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:02.771319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc90dea-51db-49da-867f-421b59df4606 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:02.798061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1102/2244] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:03.721921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:03.725181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:03.740458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b93dbb-09a0-4ccf-961f-c76eea80e630 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:03.741393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b93dbb-09a0-4ccf-961f-c76eea80e630 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:03.741829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b93dbb-09a0-4ccf-961f-c76eea80e630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:03.758633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ae539-37dd-415c-acd8-9d22b5f74a6b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:03.776301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:03.780128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:03.788332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:03.789299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] PATH_INFO: `/v3/projects/50afbcabe0ee43a1a2a499b57bf0cde0` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:03.793157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50afbcabe0ee43a1a2a499b57bf0cde0)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:03.801339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:03.821433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-52b93dbb-09a0-4ccf-961f-c76eea80e630 None None] MFA Rules not processed for user `c9074c8e166b4cd2a3cff41f0ccca273`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:03.824522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-52b93dbb-09a0-4ccf-961f-c76eea80e630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:03.922498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:03.961610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50afbcabe0ee43a1a2a499b57bf0cde0', 'request_id': 'req-1efe7285-d5f3-451f-bfb8-28fb9177c347'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:03.989780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50afbcabe0ee43a1a2a499b57bf0cde0', 'request_id': 'req-1efe7285-d5f3-451f-bfb8-28fb9177c347'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:04.007604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50afbcabe0ee43a1a2a499b57bf0cde0', 'request_id': 'req-1efe7285-d5f3-451f-bfb8-28fb9177c347'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:04.039948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] The token cache is being invalidate because project 50afbcabe0ee43a1a2a499b57bf0cde0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:07:04.040560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:04.042855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:04.049114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1143/2245] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:07:03 2022] DELETE /identity/v3/projects/50afbcabe0ee43a1a2a499b57bf0cde0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:07:04.134088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1103/2246] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:04.159958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:04.162208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:04.371827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7285-d5f3-451f-bfb8-28fb9177c347 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:04.381784 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:04.382735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:04.383124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:04.383503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:04.389627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:04.498835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:04.535730 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:04.537345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:04.552321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1144/2247] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:08.307712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d9751-57f9-4e4b-9e87-6599c2318ce1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:08.308297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d9751-57f9-4e4b-9e87-6599c2318ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:08.308544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d9751-57f9-4e4b-9e87-6599c2318ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:08.338157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-bd3d9751-57f9-4e4b-9e87-6599c2318ce1 None None] MFA Rules not processed for user `c9074c8e166b4cd2a3cff41f0ccca273`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:08.339623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bd3d9751-57f9-4e4b-9e87-6599c2318ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:08.358862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1104/2248] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:13.661026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:13.663348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:13.767468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fd898-b0a7-49f3-a432-53ef914b3ecc service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:13.772329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:13.773199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:13.773478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:13.773752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] PATH_INFO: `/v3/users/8060ee5deacf427fbfb5b278d315d6db` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:13.774959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8060ee5deacf427fbfb5b278d315d6db)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:13.776897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:13.828207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8060ee5deacf427fbfb5b278d315d6db'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:13.837712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8060ee5deacf427fbfb5b278d315d6db'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:13.863957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8060ee5deacf427fbfb5b278d315d6db'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:13.864774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8060ee5deacf427fbfb5b278d315d6db'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:13.866483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8060ee5deacf427fbfb5b278d315d6db'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:13.891239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1145/2249] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:07:13 2022] DELETE /identity/v3/users/8060ee5deacf427fbfb5b278d315d6db => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:07:13.892934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496a1d-bae3-4674-a0fa-8a21b57828e4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:13.893434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496a1d-bae3-4674-a0fa-8a21b57828e4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:13.893525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3496a1d-bae3-4674-a0fa-8a21b57828e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:13.917020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d3496a1d-bae3-4674-a0fa-8a21b57828e4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:13.923792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d3496a1d-bae3-4674-a0fa-8a21b57828e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:14.070728 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1105/2250] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:14.071199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:14.073058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:14.259838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945e59f-68a7-4a53-86c1-a6610b5cb2f0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:14.277439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:14.278486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:14.279029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:14.280208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:14.327165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:14.329171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:14.346400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1146/2251] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:14.596839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afa8a3-c3e2-4f2b-bdc2-977b6c2cdafb None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:14.597285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afa8a3-c3e2-4f2b-bdc2-977b6c2cdafb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:14.597574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afa8a3-c3e2-4f2b-bdc2-977b6c2cdafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:14.616211 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-64afa8a3-c3e2-4f2b-bdc2-977b6c2cdafb None None] MFA Rules not processed for user `0c321d34fcce4054849926e924b3c0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:14.617661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-64afa8a3-c3e2-4f2b-bdc2-977b6c2cdafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:14.678244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1106/2252] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:14.826229 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:14.827786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:14.844991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:14.861386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e21a-7ba6-4b86-8b21-aa0a0761d0ce service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:14.867814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:14.868722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:14.869006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:14.869390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] PATH_INFO: `/v3/projects/b33925410ec54e1c85163c26c95b9e14` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:14.871257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33925410ec54e1c85163c26c95b9e14)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:14.873235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:14.922167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33925410ec54e1c85163c26c95b9e14', 'request_id': 'req-886347d9-eb2f-48d6-bb13-1f3e7566013e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:14.929442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33925410ec54e1c85163c26c95b9e14', 'request_id': 'req-886347d9-eb2f-48d6-bb13-1f3e7566013e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:14.935644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33925410ec54e1c85163c26c95b9e14', 'request_id': 'req-886347d9-eb2f-48d6-bb13-1f3e7566013e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:14.949847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] The token cache is being invalidate because project b33925410ec54e1c85163c26c95b9e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:07:14.950275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:14.951042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-886347d9-eb2f-48d6-bb13-1f3e7566013e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:14.952722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1147/2253] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:07:14 2022] DELETE /identity/v3/projects/b33925410ec54e1c85163c26c95b9e14 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:07:14.957555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:14.958056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:14.958138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:14.976452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:14.977685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:15.039284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1107/2254] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:07:15.047014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.047308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.047592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.066852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:15.068066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:15.076066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1148/2255] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:15.080271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.090735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b516fbb-b7ae-4e14-bca8-ac03730577c1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.095626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.096880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.097169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.097443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.098186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:07:15.098491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.100134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.101789 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:15.116834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1108/2256] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:07:15.118208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.128320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587e324-adb4-4aac-84a3-7823532958a6 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.133235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.134358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.134358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.134590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.135157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.137063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.163618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1149/2257] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:07:15 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:07:15.167845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.179510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550c8e8-068e-45ab-803d-94a69f74f2d4 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.239801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.240488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.240939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.241053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.241821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.243876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.263755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1109/2258] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:07:15 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:07:15.268912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.282705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44945e34-fdaf-45d2-8ca6-1d4c37d31ba0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.287707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.288461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.288979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.289331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.290151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.291904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.303493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1150/2259] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:07:15.307605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.317559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895c1f-372e-49f9-af2a-13a09e2ae177 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.322872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b869e-abee-461a-9103-8feb9cd2d2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.323541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b869e-abee-461a-9103-8feb9cd2d2ec admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.323828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b869e-abee-461a-9103-8feb9cd2d2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.324108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b869e-abee-461a-9103-8feb9cd2d2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.324876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b869e-abee-461a-9103-8feb9cd2d2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.327166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b869e-abee-461a-9103-8feb9cd2d2ec admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.339417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1110/2260] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:07:15.343776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.361990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598b4fb-17f1-4737-b8b4-cfb86c3631c6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.367395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.368481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.368839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.369193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] PATH_INFO: `/v3/projects/9e422d8e74a045c49ad2c073eba88bef/users/1c6889b5f06d47d7ba9d76231ede0326/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.388843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e422d8e74a045c49ad2c073eba88bef, user_id=1c6889b5f06d47d7ba9d76231ede0326, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.400130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.435669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1151/2261] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:07:15 2022] PUT /identity/v3/projects/9e422d8e74a045c49ad2c073eba88bef/users/1c6889b5f06d47d7ba9d76231ede0326/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:07:15.446284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a04d-d048-44d4-8e38-5b4873f69575 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.446578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a04d-d048-44d4-8e38-5b4873f69575 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.446893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41a04d-d048-44d4-8e38-5b4873f69575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.492025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-9f41a04d-d048-44d4-8e38-5b4873f69575 None None] MFA Rules not processed for user `1c6889b5f06d47d7ba9d76231ede0326`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:15.496202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9f41a04d-d048-44d4-8e38-5b4873f69575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:15.645759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1111/2262] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:15.659811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:15.662938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:15.761862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8025246-6e05-4bba-9828-933527ea5972 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:15.765885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:15.767192 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:15.767508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:15.767847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:15.778391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:15.780191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:15.807156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1152/2263] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:18.970609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:18.971190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:18.971272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:18.993873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:18.995141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:19.092894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1112/2264] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:07:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:07:19.107969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:19.109816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:19.176116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:19.270247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384f720-86ec-4f6f-9857-a22b2b28ea62 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:19.279413 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77204b93-2e95-4286-83ea-ab27fc2222d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:19.280291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77204b93-2e95-4286-83ea-ab27fc2222d6 service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:19.280789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77204b93-2e95-4286-83ea-ab27fc2222d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:19.281143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77204b93-2e95-4286-83ea-ab27fc2222d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:19.291231 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77204b93-2e95-4286-83ea-ab27fc2222d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:19.292980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77204b93-2e95-4286-83ea-ab27fc2222d6 service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:19.302034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1153/2265] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:07:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:07:26.280473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:26.296130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:26.423395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a0c4-dd86-450b-9d89-e1e94aaada45 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:26.428620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:26.429640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:26.430118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:26.430550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] PATH_INFO: `/v3/users/1c6889b5f06d47d7ba9d76231ede0326` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:26.431916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6889b5f06d47d7ba9d76231ede0326)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:26.434341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:26.507443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6889b5f06d47d7ba9d76231ede0326'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:26.515042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6889b5f06d47d7ba9d76231ede0326'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:26.528381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6889b5f06d47d7ba9d76231ede0326'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:26.529055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6889b5f06d47d7ba9d76231ede0326'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:26.529663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6889b5f06d47d7ba9d76231ede0326'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:26.546862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1113/2266] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:07:26 2022] DELETE /identity/v3/users/1c6889b5f06d47d7ba9d76231ede0326 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:07:26.551518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:26.552088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:26.552088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:26.570770 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:07:26.572013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:26.646324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1154/2267] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:07:26.657111 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:26.658736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:26.722104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263d25-9c67-4972-b82b-715919990820 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:26.726112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:26.726907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:26.727290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:26.727641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:26.737961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:26.739446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:26.749108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1114/2268] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:27.049278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:27.050806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:27.071656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b021e7d-1e94-4150-ad91-e2d6fa187004 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:27.076614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:27.077587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:27.077873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:27.078226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] PATH_INFO: `/v3/projects/9e422d8e74a045c49ad2c073eba88bef` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:27.079685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e422d8e74a045c49ad2c073eba88bef)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:27.087815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:27.152019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e422d8e74a045c49ad2c073eba88bef', 'request_id': 'req-9dedafe6-8f9e-4fac-a292-0520e2f2c009'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:27.160337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e422d8e74a045c49ad2c073eba88bef', 'request_id': 'req-9dedafe6-8f9e-4fac-a292-0520e2f2c009'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:27.166163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e422d8e74a045c49ad2c073eba88bef', 'request_id': 'req-9dedafe6-8f9e-4fac-a292-0520e2f2c009'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:27.181521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] The token cache is being invalidate because project 9e422d8e74a045c49ad2c073eba88bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:07:27.181821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:27.182488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:07:27.194241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1155/2269] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:07:27 2022] DELETE /identity/v3/projects/9e422d8e74a045c49ad2c073eba88bef => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:07:36.744711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=76,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:37.177509 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:07:44.109909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:07:44.111428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:44.183049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683a6a9-4a71-4db6-8a05-115c0bff5899 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:07:44.245036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:07:44.245696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:07:44.245979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:07:44.246251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:07:44.248105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:07:44.259759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:07:44.261341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dc172-fdff-40d9-b356-9509480a968f service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:07:44.274452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1115/2270] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:07:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:07:54.275490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=58 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:08:00.910133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:08:00.911680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:08:00.978368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedafe6-8f9e-4fac-a292-0520e2f2c009 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:08:00.983866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:08:00.984523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:08:00.984804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:08:00.985079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:08:00.986934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:08:01.054199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:08:01.056077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af299b44-e486-4bb0-a7ff-d83db46ec72f service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:08:01.066019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1156/2271] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:08:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:08:11.061657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=50 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:08:11.324885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:08:11.325378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:08:11.325488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:08:11.345022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] MFA Rules not processed for user `d9cab6f67cd34c46886d036ba170331e`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:08:11.346370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:08:11.405332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1116/2272] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:08:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:08:21.398287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=25 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:08:49.320598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4a9b-f2e2-46d2-8f4f-a978afb8fe6a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:08:49.321061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4a9b-f2e2-46d2-8f4f-a978afb8fe6a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:08:49.321061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba4a9b-f2e2-46d2-8f4f-a978afb8fe6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:08:49.344273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-65ba4a9b-f2e2-46d2-8f4f-a978afb8fe6a None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:08:49.345637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-65ba4a9b-f2e2-46d2-8f4f-a978afb8fe6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:08:49.410955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1157/2273] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:08:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:08:49.450437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:08:49.452478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:08:49.528858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e8bbf-94c2-4230-91d0-95864664ac96 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:08:49.536002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff3af5-d006-4d5b-a26d-6cd05b60ddd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:08:49.536612 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff3af5-d006-4d5b-a26d-6cd05b60ddd9 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:08:49.537008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff3af5-d006-4d5b-a26d-6cd05b60ddd9 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:08:49.537431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff3af5-d006-4d5b-a26d-6cd05b60ddd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:08:49.546823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff3af5-d006-4d5b-a26d-6cd05b60ddd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:08:49.548721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff3af5-d006-4d5b-a26d-6cd05b60ddd9 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:08:49.575817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1117/2274] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:08:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:08:59.404247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=25 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:08:59.554676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=32 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:09:04.881917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fc313-338c-461a-90b1-4589554c4e58 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:04.882390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fc313-338c-461a-90b1-4589554c4e58 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:04.882504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fc313-338c-461a-90b1-4589554c4e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:04.900307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b89fc313-338c-461a-90b1-4589554c4e58 None None] MFA Rules not processed for user `c9074c8e166b4cd2a3cff41f0ccca273`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:09:04.901485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b89fc313-338c-461a-90b1-4589554c4e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:04.953878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1158/2275] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:09:08.416271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24726e92-6dd1-46c4-b0a2-5c7ce1debc09 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:08.416768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24726e92-6dd1-46c4-b0a2-5c7ce1debc09 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:08.417034 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24726e92-6dd1-46c4-b0a2-5c7ce1debc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:08.435461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-24726e92-6dd1-46c4-b0a2-5c7ce1debc09 None None] MFA Rules not processed for user `dc40c07fceab4a83ba9324da868034aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:09:08.436726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-24726e92-6dd1-46c4-b0a2-5c7ce1debc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:08.492816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1118/2276] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:09:11.385209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631810f-596f-4504-b5e3-733acd866d39 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.385786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631810f-596f-4504-b5e3-733acd866d39 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.390751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631810f-596f-4504-b5e3-733acd866d39 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.393109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1159/2277] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:09:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:09:11.412574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f757-20a5-42cb-9ec8-ed2c460a4958 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.413146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f757-20a5-42cb-9ec8-ed2c460a4958 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.413146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f757-20a5-42cb-9ec8-ed2c460a4958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.452137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0532f757-20a5-42cb-9ec8-ed2c460a4958 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:09:11.453368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0532f757-20a5-42cb-9ec8-ed2c460a4958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:11.550582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1119/2278] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:09:11.559190 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631810f-596f-4504-b5e3-733acd866d39 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:11.561525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7631810f-596f-4504-b5e3-733acd866d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:11.642717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631810f-596f-4504-b5e3-733acd866d39 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:11.646466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7b019-69b7-4623-84f1-c8cff2a6e0db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:11.647224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b019-69b7-4623-84f1-c8cff2a6e0db service glance] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.647718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b019-69b7-4623-84f1-c8cff2a6e0db service glance] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.648004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7b019-69b7-4623-84f1-c8cff2a6e0db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.659273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b019-69b7-4623-84f1-c8cff2a6e0db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:11.660733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7b019-69b7-4623-84f1-c8cff2a6e0db service glance] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:11.672707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1160/2279] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:09:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:09:11.744713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaaa27-1f49-4ede-842a-5cf3a4089af5 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.745058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaaa27-1f49-4ede-842a-5cf3a4089af5 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.745359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaaa27-1f49-4ede-842a-5cf3a4089af5 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.746777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1120/2280] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:09:11.751905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f2f6e-f0ef-47d4-b9a7-ab7a0398c7fe None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.752273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f2f6e-f0ef-47d4-b9a7-ab7a0398c7fe None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.752622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f2f6e-f0ef-47d4-b9a7-ab7a0398c7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.772063 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-f16f2f6e-f0ef-47d4-b9a7-ab7a0398c7fe None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:09:11.773335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f16f2f6e-f0ef-47d4-b9a7-ab7a0398c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:11.780359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1161/2281] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:09:11.794420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaaa27-1f49-4ede-842a-5cf3a4089af5 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:11.796701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4dbaaa27-1f49-4ede-842a-5cf3a4089af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:11.813611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbaaa27-1f49-4ede-842a-5cf3a4089af5 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:11.818837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac122af-758e-4135-ad87-20f75e235465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:11.819482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac122af-758e-4135-ad87-20f75e235465 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.819768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac122af-758e-4135-ad87-20f75e235465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.820032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac122af-758e-4135-ad87-20f75e235465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.829349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac122af-758e-4135-ad87-20f75e235465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:11.830909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac122af-758e-4135-ad87-20f75e235465 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:11.840271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1121/2282] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:09:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:09:11.969380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec2f35-8b50-4d6c-9fa8-ef1a3f2f54d8 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.969744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec2f35-8b50-4d6c-9fa8-ef1a3f2f54d8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.970099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec2f35-8b50-4d6c-9fa8-ef1a3f2f54d8 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:11.971729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1162/2283] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:09:11.977907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d689e3-0b0e-43a4-a1e5-ba6e8d0ab391 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:11.978478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d689e3-0b0e-43a4-a1e5-ba6e8d0ab391 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:11.979001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d689e3-0b0e-43a4-a1e5-ba6e8d0ab391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:12.001115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f8d689e3-0b0e-43a4-a1e5-ba6e8d0ab391 None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:09:12.003259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f8d689e3-0b0e-43a4-a1e5-ba6e8d0ab391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:12.015397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1122/2284] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:09:12.034539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec2f35-8b50-4d6c-9fa8-ef1a3f2f54d8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:12.047839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec2f35-8b50-4d6c-9fa8-ef1a3f2f54d8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:12.051613 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:12.052756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:12.053040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:12.053317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:12.065458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:12.067059 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:12.078754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1163/2285] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:09:21.993361 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=65 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:09:22.074153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=60 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:09:23.264592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:23.265070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:23.270227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:23.290013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:09:23.291263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:23.303702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1123/2286] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:09:23.316324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:23.329315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647fa2c-5346-49b1-ac91-4e956a288b18 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:23.335202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:23.335864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:23.336155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:23.336438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:23.345813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:23.347543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:23.356954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1164/2287] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:09:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:09:31.908408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:31.918877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32ab60-c865-4a16-95ae-5ee7b50dccb3 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:31.922488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:31.923161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:31.923455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:31.923736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:31.925559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:31.940909 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:31.942871 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:31.953896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1124/2288] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:09:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:09:32.611971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:32.622307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafd567-e057-435d-ad1c-02f7b49a06f4 service swift] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:32.625941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:32.626578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:32.626904 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:32.627180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:32.628926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:32.643488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:32.644894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63222f-68fa-44ad-b678-8dd8650d9bed service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:32.655461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1165/2289] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:09:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:09:41.948873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=9 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:09:42.650736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=12 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:09:49.499180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:09:49.500678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:49.514623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7726d3-ac74-4d35-88de-bfeaa95e6f5d service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:09:49.520340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:09:49.520969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:09:49.521903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:09:49.522180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:09:49.523888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:09:49.581665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:09:49.583227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:09:49.592139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1125/2290] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:09:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:09:59.590146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=28 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:10:07.419391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:07.419891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:07.419988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:07.437129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:10:07.438361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:07.446405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1166/2291] 10.222.0.39 () {58 vars in 967 bytes} [Wed Jul 13 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:10:07.468983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:10:07.480169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fe531-104a-4736-b667-3b82b0c8d103 service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:10:07.485224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:10:07.485860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:07.486142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:07.486420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:07.494434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:10:07.496059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:10:07.503957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1126/2292] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:10:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:10:17.432100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:10:17.500717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:10:24.100438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:10:24.110971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5b6be-f694-4c68-a0f4-4f885e88b74e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:10:24.114881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:10:24.115563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:24.115848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:24.116127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:24.118696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:24.130908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:10:24.132417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:10:24.142010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1167/2293] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:10:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:10:34.137426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:10:41.779864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:10:41.781985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:41.855537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a1ef8-c464-40a7-bc0e-d8b1c31813ca service swift] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:10:41.860086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:10:41.860954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:41.861240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:41.861520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] PATH_INFO: `/v3/users/c9074c8e166b4cd2a3cff41f0ccca273` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:41.862747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9074c8e166b4cd2a3cff41f0ccca273)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:10:41.864677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:10:41.912220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9074c8e166b4cd2a3cff41f0ccca273'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:41.924437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9074c8e166b4cd2a3cff41f0ccca273'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:41.938705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9074c8e166b4cd2a3cff41f0ccca273'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:41.939408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9074c8e166b4cd2a3cff41f0ccca273'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:41.940020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9074c8e166b4cd2a3cff41f0ccca273'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:41.952983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1127/2294] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:10:41 2022] DELETE /identity/v3/users/c9074c8e166b4cd2a3cff41f0ccca273 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:10:42.136144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:10:42.155765 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac943a-ec35-4467-9bc5-578cb1a92214 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:10:42.220942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:10:42.221810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:42.222094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:42.222374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] PATH_INFO: `/v3/projects/2332a661874a4a2faebb32a154be663c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:42.223655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2332a661874a4a2faebb32a154be663c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:10:42.225602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:10:42.274780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2332a661874a4a2faebb32a154be663c', 'request_id': 'req-30435931-4666-4680-8b6d-97282178a9c7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:42.282490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2332a661874a4a2faebb32a154be663c', 'request_id': 'req-30435931-4666-4680-8b6d-97282178a9c7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:42.287883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2332a661874a4a2faebb32a154be663c', 'request_id': 'req-30435931-4666-4680-8b6d-97282178a9c7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:42.301341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] The token cache is being invalidate because project 2332a661874a4a2faebb32a154be663c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:10:42.301341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:42.301952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:10:42.304047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1168/2295] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:10:42 2022] DELETE /identity/v3/projects/2332a661874a4a2faebb32a154be663c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:10:51.949293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:10:52.298162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:10:55.402224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:10:55.403683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:55.475293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1db8-8edf-4636-8dae-fb36c6396ef7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:10:55.535778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:10:55.536359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:55.536643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:55.536956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:55.539267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:55.594108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:10:55.595706 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:10:55.605118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1128/2296] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:10:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:10:59.488239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:10:59.489556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:59.504859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30435931-4666-4680-8b6d-97282178a9c7 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:10:59.510212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:10:59.510872 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:10:59.511151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:10:59.511446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:10:59.513121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:10:59.569885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:10:59.571490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2b06d-2c1a-405c-b029-00b289244774 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:10:59.580271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1169/2297] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:11:03.083530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:03.085078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:03.105951 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee2867-c7a3-43c8-94c6-ea32155b0e11 service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:03.112674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:03.113665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:03.114234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:03.115105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:03.120764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:03.135103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:03.136779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:03.148688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1129/2298] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:11:09.576222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=28 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:12.973411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:12.973999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:12.973999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:12.991824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] MFA Rules not processed for user `d9cab6f67cd34c46886d036ba170331e`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:11:12.993030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:12.999877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1170/2299] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:11:13.143538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:13.305725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:13.307159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:13.363297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb01fca-5965-41b3-8ded-c0c042c5935b service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:13.367605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:13.368422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:13.368603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:13.368863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:13.370727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:13.380541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:13.381972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:13.390272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1130/2300] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:11:13.702543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:13.704100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:13.776051 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595054-2c86-46a9-b087-7bb63001bf70 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:13.780903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:13.781544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:13.781825 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:13.782099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:13.783943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:13.794929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:13.796522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:13.806079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1171/2301] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:11:23.386508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=26 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:23.802017 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=35 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:25.330928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:25.344636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb8238-63d3-4203-94ad-cc418e46df22 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:25.351384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:25.352118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:25.352332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:25.352652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:25.354547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:25.380177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:25.381882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c1ba6-eff6-4095-a11b-d2a2fbbabc26 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:25.394543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1131/2302] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:11:30.102943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:30.114883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd34e9-1c53-4a51-966f-774e941e3643 service placement] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:30.121049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:30.121699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:30.121982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:30.122257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:30.124093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:30.136974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:30.138822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:30.149160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1172/2303] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:11:35.388479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:37.280527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:37.281095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:37.281095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:37.298426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] MFA Rules not processed for user `0884a33a47e448e7826f257880a28964`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:11:37.299849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:37.354560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1132/2304] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:11:37.657497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:37.658947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:37.670668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5ccfd-8e1f-4cff-af52-96cd5fe1a12f service nova] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:37.674175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:37.674844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:37.675133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:37.675409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:37.677089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:37.686096 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:37.687544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7859b-451a-47df-9855-8426d4e9e2bd service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:37.695381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1173/2305] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:11:47.347949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=25 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:47.691800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=12 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:11:54.614249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:54.628340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f8fce-fdd4-4e6e-9a97-5200149df697 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:54.634355 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:54.635047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:54.635330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:54.635606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:54.637690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:54.699943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:54.701694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:54.711795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1133/2306] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:11:58.291996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:58.292456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:58.292553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:58.311055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:11:58.312245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:58.365266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1174/2307] 10.222.0.39 () {58 vars in 967 bytes} [Wed Jul 13 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:11:58.411802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:58.413878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:58.484784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e288a8-4c64-46e8-8ee7-10a57ca240c1 service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:58.490123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b17bd-8f83-4325-bba0-12198950a3f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:58.490843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b17bd-8f83-4325-bba0-12198950a3f7 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:58.491128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b17bd-8f83-4325-bba0-12198950a3f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:58.491406 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b17bd-8f83-4325-bba0-12198950a3f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:58.500700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b17bd-8f83-4325-bba0-12198950a3f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:58.502349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b17bd-8f83-4325-bba0-12198950a3f7 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:58.512302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1134/2308] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:11:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:11:58.854742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:11:58.863855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73742e66-6918-47c5-bc05-0786f9fe4b13 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:11:58.867342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:11:58.867978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:11:58.868263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:11:58.868539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:11:58.870308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:11:58.933792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:11:58.935250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:11:58.945245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1175/2309] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:12:00.001215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241310c1-bbb9-415e-91f0-11ed45982d5e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:00.001786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241310c1-bbb9-415e-91f0-11ed45982d5e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:00.001786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241310c1-bbb9-415e-91f0-11ed45982d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:00.021049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-241310c1-bbb9-415e-91f0-11ed45982d5e None None] MFA Rules not processed for user `dc40c07fceab4a83ba9324da868034aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:12:00.022795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-241310c1-bbb9-415e-91f0-11ed45982d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:00.029918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1135/2310] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:12:02.263636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:12:02.265154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:02.284401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740a3bb-9271-47b5-8f77-70feea18a739 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:12:02.293570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:12:02.295115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:02.295595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:02.295968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:02.298775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:02.315216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:12:02.316887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd719b3a-f85f-4f3f-afbb-be6e3d489f66 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:12:02.330636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1176/2311] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:12:05.456778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6d4fd-9466-4a64-a80f-02a4105e5632 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:05.456778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6d4fd-9466-4a64-a80f-02a4105e5632 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:05.456778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6d4fd-9466-4a64-a80f-02a4105e5632 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:05.458167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1136/2312] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:12:05.472015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:05.472565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:05.472565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:05.492644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] MFA Rules not processed for user `ab905149e23347d0aed37819978ac06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:12:05.493941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:05.502522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1177/2313] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:12:05.519044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6d4fd-9466-4a64-a80f-02a4105e5632 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:12:05.541726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6d4fd-9466-4a64-a80f-02a4105e5632 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:12:05.547805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:12:05.548576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:05.548883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:05.549443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:05.559041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:12:05.560841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:12:05.577479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1137/2314] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:12:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:12:05.592313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:12:05.593727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:05.608068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6977e3d-cc97-4d7e-af44-8ef868b269a5 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:12:05.612861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:12:05.614401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:05.614721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:05.615213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] PATH_INFO: `/v3/users/0c321d34fcce4054849926e924b3c0a8` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:05.618158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c321d34fcce4054849926e924b3c0a8)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:12:05.620257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:12:05.670109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c321d34fcce4054849926e924b3c0a8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:05.677882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c321d34fcce4054849926e924b3c0a8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:05.690934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c321d34fcce4054849926e924b3c0a8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:05.692069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c321d34fcce4054849926e924b3c0a8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:05.692681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c321d34fcce4054849926e924b3c0a8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:05.694719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:12:05.709286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1178/2315] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:12:05 2022] DELETE /identity/v3/users/0c321d34fcce4054849926e924b3c0a8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:12:05.883624 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:12:05.895866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b266a-8ef7-401a-b764-fa89d57f8f5c service placement] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:12:05.961232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:12:05.962098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:05.962487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:05.962804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] PATH_INFO: `/v3/projects/c09f02b3463a47e68081b8b47cc86985` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:05.963991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09f02b3463a47e68081b8b47cc86985)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:12:05.965911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:12:05.977702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:12:06.015153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09f02b3463a47e68081b8b47cc86985', 'request_id': 'req-112632b5-17b8-4e6f-819e-cee085fb5ba0'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.021191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09f02b3463a47e68081b8b47cc86985', 'request_id': 'req-112632b5-17b8-4e6f-819e-cee085fb5ba0'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.027609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09f02b3463a47e68081b8b47cc86985', 'request_id': 'req-112632b5-17b8-4e6f-819e-cee085fb5ba0'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.042048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] The token cache is being invalidate because project c09f02b3463a47e68081b8b47cc86985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:12:06.042338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.042982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.044508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1138/2316] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:12:05 2022] DELETE /identity/v3/projects/c09f02b3463a47e68081b8b47cc86985 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:12:06.048453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:12:06.049978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:06.063503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bbccf-758f-4a32-9071-c90ed6bec1b3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:12:06.123680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:12:06.124558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:06.124839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:06.125112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] PATH_INFO: `/v3/users/33bd44d3ce8e479eaaae9c1145042ff8` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:06.126289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bd44d3ce8e479eaaae9c1145042ff8)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:12:06.128239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:12:06.177090 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bd44d3ce8e479eaaae9c1145042ff8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.187085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bd44d3ce8e479eaaae9c1145042ff8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.200640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bd44d3ce8e479eaaae9c1145042ff8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.201539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bd44d3ce8e479eaaae9c1145042ff8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.202317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-661dd20c-f61b-43c9-94f9-d91b6cb19320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bd44d3ce8e479eaaae9c1145042ff8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.215259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1179/2317] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:12:06 2022] DELETE /identity/v3/users/33bd44d3ce8e479eaaae9c1145042ff8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:12:06.393259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:12:06.396343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:06.414799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112632b5-17b8-4e6f-819e-cee085fb5ba0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:12:06.481788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:12:06.482805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:06.483203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:06.483536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] PATH_INFO: `/v3/projects/efdf91ab1557410db294ec94a183842f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:06.484809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efdf91ab1557410db294ec94a183842f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:12:06.486964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:12:06.538337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efdf91ab1557410db294ec94a183842f', 'request_id': 'req-09f72d51-37a3-4014-9b68-eda457cf5c43'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.557440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efdf91ab1557410db294ec94a183842f', 'request_id': 'req-09f72d51-37a3-4014-9b68-eda457cf5c43'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.564418 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efdf91ab1557410db294ec94a183842f', 'request_id': 'req-09f72d51-37a3-4014-9b68-eda457cf5c43'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.583411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] The token cache is being invalidate because project efdf91ab1557410db294ec94a183842f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:12:06.584021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.585861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:12:06.593757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1139/2318] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:12:06 2022] DELETE /identity/v3/projects/efdf91ab1557410db294ec94a183842f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:12:16.211719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:12:16.577615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=71,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:12:30.601926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:12:30.602468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:12:30.602575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:12:30.619288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] MFA Rules not processed for user `fc4afeb23348438eabfe22f1a975f9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:12:30.620522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:12:30.673629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1180/2319] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:12:40.667047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=25 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:14.340495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:14.344588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:14.420469 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:14.421999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:14.581472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fb054-baea-4a7f-932a-252a4faf0a60 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:14.587586 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:14.588396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72d51-37a3-4014-9b68-eda457cf5c43 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:14.588541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:14.588933 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:14.589310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:14.591677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:14.595001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:14.595334 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:14.595618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:14.595892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:14.605722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:14.607479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:14.607626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1eee0-7ba4-45ee-8579-ebf31da41147 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:14.609164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:14.620154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1181/2320] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:13:14.633618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1140/2321] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:18.385030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:18.385507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:18.385614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:18.402827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] MFA Rules not processed for user `d9cab6f67cd34c46886d036ba170331e`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:18.404015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:18.457398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1182/2322] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:24.616137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=63 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:26.010021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:26.011484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:26.094302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407715a-6fe8-441e-a460-414ed080c001 service placement] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:26.097994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:26.098666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:26.098959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:26.099238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:26.100997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:26.187719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:26.189150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:26.207432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1141/2323] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:13:28.450187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=79 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:29.521966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:29.524525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:29.541391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864f262-5966-486e-8f7d-2325177692b8 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:29.547506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:29.548534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:29.548817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:29.549099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] PATH_INFO: `/v3/users/dc40c07fceab4a83ba9324da868034aa` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:29.551492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc40c07fceab4a83ba9324da868034aa)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:29.553875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:29.624650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc40c07fceab4a83ba9324da868034aa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:29.634228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc40c07fceab4a83ba9324da868034aa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:29.649928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc40c07fceab4a83ba9324da868034aa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:29.650737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc40c07fceab4a83ba9324da868034aa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:29.651350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-0d14cb2c-c16a-4f32-a427-b69125204861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc40c07fceab4a83ba9324da868034aa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:29.665651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1183/2324] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:29 2022] DELETE /identity/v3/users/dc40c07fceab4a83ba9324da868034aa => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:29.926311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:29.937085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0018059-c93e-4fb2-ab2d-ee732c600476 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.009382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.010248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.010527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.010843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] PATH_INFO: `/v3/projects/c1d4119a1d0b474fb568818f1aaa933b` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.012024 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d4119a1d0b474fb568818f1aaa933b)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.015266 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.021469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:30.068314 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d4119a1d0b474fb568818f1aaa933b', 'request_id': 'req-5ca3fc4a-6b65-457c-8335-30a921029214'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:30.075912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d4119a1d0b474fb568818f1aaa933b', 'request_id': 'req-5ca3fc4a-6b65-457c-8335-30a921029214'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:30.081558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d4119a1d0b474fb568818f1aaa933b', 'request_id': 'req-5ca3fc4a-6b65-457c-8335-30a921029214'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:30.093357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] The token cache is being invalidate because project c1d4119a1d0b474fb568818f1aaa933b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:30.093727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:30.094434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5ca3fc4a-6b65-457c-8335-30a921029214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:30.099363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1142/2325] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:29 2022] DELETE /identity/v3/projects/c1d4119a1d0b474fb568818f1aaa933b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:30.101589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.101868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.102152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.160243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:30.161397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:30.219656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1184/2326] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:30.227840 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.228133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.228417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.246944 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:30.248165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:30.256304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1143/2327] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:30.260680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.270287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64538680-6e55-4dba-8a00-12ce8ac752e5 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.275106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.275766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.276052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.276330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.277068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:13:30.277375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.279004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.291191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1185/2328] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:13:30.295429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.305316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e280008-9f2d-4a31-ba89-6df8449a845a None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.309964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.310607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.310920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.311195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.311890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.313769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.337947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1144/2329] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:30 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:30.341952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.352829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee07d5c-06b8-454a-8aef-0c27fa5ca3a5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.380078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:30.411951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.412569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.412849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.413123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.414298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.416225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.435453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1186/2330] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:30 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:30.439504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.449708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4a25d-ed49-4812-9bfa-88eef01cf9bb admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.454261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.454971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.455253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.455526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.456279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.457861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.468595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1145/2331] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:30.472244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.482762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07c4d6-0c1d-4214-92c7-d7404aa86375 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.487377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7bbbd-8c1c-4e25-a567-5362a2c816c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.488021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7bbbd-8c1c-4e25-a567-5362a2c816c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.488305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7bbbd-8c1c-4e25-a567-5362a2c816c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.488582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7bbbd-8c1c-4e25-a567-5362a2c816c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.489303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7bbbd-8c1c-4e25-a567-5362a2c816c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.490916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7bbbd-8c1c-4e25-a567-5362a2c816c6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.500590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1187/2332] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:30.504398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.514421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a79ee-bbf7-4a3e-b41e-89734ec260e4 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.519035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.519896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.520180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.520463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] PATH_INFO: `/v3/projects/ac0722fdade14d0ab5dcdbeb33ea4a11/users/c2c1e768cd7247de9895b492550af64a/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.535130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0722fdade14d0ab5dcdbeb33ea4a11, user_id=c2c1e768cd7247de9895b492550af64a, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.537781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.551967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1146/2333] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:30 2022] PUT /identity/v3/projects/ac0722fdade14d0ab5dcdbeb33ea4a11/users/c2c1e768cd7247de9895b492550af64a/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:30.561692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.562144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.562260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.591496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] MFA Rules not processed for user `c2c1e768cd7247de9895b492550af64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:30.593023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:30.658755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1188/2334] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:30.670752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:30.672392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:30.762182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:30.773323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399070ae-e69c-4829-9ed0-e3b0169687f5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:30.779339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca0b92-d4aa-4ca9-bb73-dc9d002edc03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:30.779990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca0b92-d4aa-4ca9-bb73-dc9d002edc03 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:30.780316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca0b92-d4aa-4ca9-bb73-dc9d002edc03 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:30.780593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca0b92-d4aa-4ca9-bb73-dc9d002edc03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:30.789135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca0b92-d4aa-4ca9-bb73-dc9d002edc03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:30.790745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca0b92-d4aa-4ca9-bb73-dc9d002edc03 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:30.800123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1147/2335] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:31.020114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:31.085274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc7c60-771b-40ce-b150-8beceaa0e97f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:31.090404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:31.091323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.091592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.091866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] PATH_INFO: `/v3/users/c2c1e768cd7247de9895b492550af64a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.093204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c1e768cd7247de9895b492550af64a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:31.095062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:31.139763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:31.146920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c1e768cd7247de9895b492550af64a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.155489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c1e768cd7247de9895b492550af64a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.168216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c1e768cd7247de9895b492550af64a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.168907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c1e768cd7247de9895b492550af64a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.169515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c1e768cd7247de9895b492550af64a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.183888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1189/2336] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:31 2022] DELETE /identity/v3/users/c2c1e768cd7247de9895b492550af64a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:31.188232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.188713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.188809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.206430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:31.207652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:31.266928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1148/2337] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:31.276207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:31.277637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:31.333672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db44fb7-4061-4b06-8992-bc001ce7c970 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:31.337353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb3288-641d-4162-bf6f-32cd605ee3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:31.337992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb3288-641d-4162-bf6f-32cd605ee3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.338276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb3288-641d-4162-bf6f-32cd605ee3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.338554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb3288-641d-4162-bf6f-32cd605ee3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.347487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb3288-641d-4162-bf6f-32cd605ee3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:31.348916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb3288-641d-4162-bf6f-32cd605ee3d6 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:31.358491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1190/2338] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:31.636575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:31.638018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:31.650293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b891c3-ac77-4b0f-a6c9-03a992ffa968 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:31.654895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:31.655762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.656095 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.656320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] PATH_INFO: `/v3/projects/ac0722fdade14d0ab5dcdbeb33ea4a11` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.657551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0722fdade14d0ab5dcdbeb33ea4a11)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:31.659505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:31.705153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0722fdade14d0ab5dcdbeb33ea4a11', 'request_id': 'req-24a906de-9d36-4bce-b123-c45da097a7c8'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.710997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0722fdade14d0ab5dcdbeb33ea4a11', 'request_id': 'req-24a906de-9d36-4bce-b123-c45da097a7c8'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.716333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0722fdade14d0ab5dcdbeb33ea4a11', 'request_id': 'req-24a906de-9d36-4bce-b123-c45da097a7c8'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.728333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] The token cache is being invalidate because project ac0722fdade14d0ab5dcdbeb33ea4a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:31.728619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.729232 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-24a906de-9d36-4bce-b123-c45da097a7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:31.730773 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1149/2339] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:31 2022] DELETE /identity/v3/projects/ac0722fdade14d0ab5dcdbeb33ea4a11 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:31.735561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.736013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.736133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.754273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:31.755500 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:31.812632 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1191/2340] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:31.820174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.820463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.820740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.838343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:31.839593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:31.847310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1150/2341] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:31.851752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:31.861176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffddbb-df03-4e09-949f-785860082f64 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:31.865785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:31.866476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.867300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.867576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.868297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:13:31.868599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:31.870176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:31.881641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1192/2342] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:13:31.886277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:31.895561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727bec5-2b7d-4cb2-a574-e2d6e955248e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:31.900253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:31.900904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:31.901188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:31.901468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:31.902170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:31.904118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:31.928915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1151/2343] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:31 2022] POST /identity/v3/projects => generated 405 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:31.933455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:31.943328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037019d-2afa-473f-9358-0b80353d595b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:31.999197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:32.017865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:32.018551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:32.018978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:32.019262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:32.020688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:32.022086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:32.058593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1193/2344] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:31 2022] POST /identity/v3/users => generated 303 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:32.063259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:32.081346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e0b34-1b12-4e87-bc36-a97553cf00c8 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:32.089661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:32.090849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:32.091330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:32.091838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:32.093243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:32.096045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:32.115500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1152/2345] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:32.119594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:32.131191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f3a8c-c730-4adc-b47a-038176a00fb9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:32.136697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc183838-d651-4329-9612-86866a74436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:32.137398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc183838-d651-4329-9612-86866a74436a admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:32.137679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc183838-d651-4329-9612-86866a74436a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:32.137952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc183838-d651-4329-9612-86866a74436a admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:32.138715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc183838-d651-4329-9612-86866a74436a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:32.140309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc183838-d651-4329-9612-86866a74436a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:32.152024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1194/2346] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:32.159802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:32.170360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc068d9d-55c5-4446-a47c-afa575847204 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:32.175081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:32.175962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:32.176248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:32.176525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] PATH_INFO: `/v3/projects/fc29b591126b4791be013ac461dddd39/users/73eb1512661a41f38efb68c588f827c8/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:32.192481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc29b591126b4791be013ac461dddd39, user_id=73eb1512661a41f38efb68c588f827c8, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:32.195252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:32.209029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1153/2347] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:32 2022] PUT /identity/v3/projects/fc29b591126b4791be013ac461dddd39/users/73eb1512661a41f38efb68c588f827c8/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:32.214671 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bb9ac-1f4b-4a8e-9bcd-f9f76a0e2819 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:32.214957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bb9ac-1f4b-4a8e-9bcd-f9f76a0e2819 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:32.215250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bb9ac-1f4b-4a8e-9bcd-f9f76a0e2819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:32.233396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-1a7bb9ac-1f4b-4a8e-9bcd-f9f76a0e2819 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:32.234598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1a7bb9ac-1f4b-4a8e-9bcd-f9f76a0e2819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:32.302187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1195/2348] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:32.311691 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:32.313414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:32.323756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:32.374718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f7df2-5dd5-4009-ab81-a7f9e21f389d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:32.378331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:32.379007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:32.379291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:32.379566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:32.389362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:32.390842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:32.401251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1154/2349] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:33.344168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.345081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.345553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.382476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] MFA Rules not processed for user `73eb1512661a41f38efb68c588f827c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:33.383992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:33.465876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1196/2350] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:33.467070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:33.484143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd489984-8f35-4d0a-b288-e22999a371e5 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:33.490857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:33.491783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.494820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.496045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.496289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:33.498340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:33.532829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1155/2351] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:33 2022] POST /identity/v3/projects => generated 405 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:13:33.534025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:33.546314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbea3a-2aee-4a1d-9cfb-f6aaf8799ac9 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:33.599700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:33.611886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:33.612569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.612859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.613141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.613839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:33.615811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:33.643001 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1197/2352] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:33 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:33.646760 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:33.658013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851eed9-152d-436c-a279-e1d6b4aae902 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:33.662843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:33.664077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.664359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.664628 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.665392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:33.667024 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:33.678929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1156/2353] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:33.683148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:33.693376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a7c84-4e8b-46c2-96e9-07b9cee7a7d5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:33.698364 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:33.699273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.699558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.699836 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] PATH_INFO: `/v3/projects/a155379c55514a56a67597939b0f4630/users/0fae5f3019824bf3b8b5d002889b0ecb/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.716496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a155379c55514a56a67597939b0f4630, user_id=0fae5f3019824bf3b8b5d002889b0ecb, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:33.719341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:33.736127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1198/2354] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:33 2022] PUT /identity/v3/projects/a155379c55514a56a67597939b0f4630/users/0fae5f3019824bf3b8b5d002889b0ecb/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:33.740480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:33.752610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90be4f-f20d-4a2a-981d-23884694ffa8 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:33.815834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e810b-974a-45bd-9917-0aaf9c3e3c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:33.816553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e810b-974a-45bd-9917-0aaf9c3e3c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.816878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e810b-974a-45bd-9917-0aaf9c3e3c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.817158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e810b-974a-45bd-9917-0aaf9c3e3c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.817895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e810b-974a-45bd-9917-0aaf9c3e3c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:33.820078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e810b-974a-45bd-9917-0aaf9c3e3c21 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:33.831636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1157/2355] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:33.836452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:33.848043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f1575-634c-4d20-bc0b-1c2d620dfdaa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:33.852702 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:33.853942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:33.853942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:33.854161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] PATH_INFO: `/v3/projects/a155379c55514a56a67597939b0f4630/users/0fae5f3019824bf3b8b5d002889b0ecb/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:33.856267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a155379c55514a56a67597939b0f4630, user_id=0fae5f3019824bf3b8b5d002889b0ecb, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:33.858963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:33.871031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1199/2356] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:33 2022] PUT /identity/v3/projects/a155379c55514a56a67597939b0f4630/users/0fae5f3019824bf3b8b5d002889b0ecb/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:34.790905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:34.791474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:34.791474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:34.819812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] MFA Rules not processed for user `0fae5f3019824bf3b8b5d002889b0ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:34.821038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:34.901054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:34.909053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1158/2357] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:35.951490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:36.015827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603d22-10e3-4dbc-baaa-af82a077ed6b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:36.020464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:36.021340 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:36.021626 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:36.021906 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] PATH_INFO: `/v3/users/73eb1512661a41f38efb68c588f827c8` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:36.023156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=73eb1512661a41f38efb68c588f827c8)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:36.025075 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:36.073119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73eb1512661a41f38efb68c588f827c8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.080048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73eb1512661a41f38efb68c588f827c8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.091889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73eb1512661a41f38efb68c588f827c8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.092547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73eb1512661a41f38efb68c588f827c8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.093163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73eb1512661a41f38efb68c588f827c8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.105918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1200/2358] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:35 2022] DELETE /identity/v3/users/73eb1512661a41f38efb68c588f827c8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:36.287430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:36.298463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952422fb-75c3-43db-9cc7-47a3ead6a7d5 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:36.355564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:36.356817 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:36.357293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:36.357631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] PATH_INFO: `/v3/projects/fc29b591126b4791be013ac461dddd39` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:36.358841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc29b591126b4791be013ac461dddd39)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:36.360741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:36.405261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc29b591126b4791be013ac461dddd39', 'request_id': 'req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.411256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc29b591126b4791be013ac461dddd39', 'request_id': 'req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.417207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc29b591126b4791be013ac461dddd39', 'request_id': 'req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.429849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] The token cache is being invalidate because project fc29b591126b4791be013ac461dddd39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:36.430133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.430788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.432423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1159/2359] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:36 2022] DELETE /identity/v3/projects/fc29b591126b4791be013ac461dddd39 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:36.436423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:36.437816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:36.450775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537debf5-96a1-40ca-91d0-38f26ca39f3e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:36.476619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:36.510018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:36.510908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:36.511187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:36.511457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] PATH_INFO: `/v3/users/0fae5f3019824bf3b8b5d002889b0ecb` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:36.512637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fae5f3019824bf3b8b5d002889b0ecb)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:36.514530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:36.560092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fae5f3019824bf3b8b5d002889b0ecb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.567070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fae5f3019824bf3b8b5d002889b0ecb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.578710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fae5f3019824bf3b8b5d002889b0ecb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.579368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fae5f3019824bf3b8b5d002889b0ecb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.579965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-079ff21c-0bc1-4691-8661-0986ab88f128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fae5f3019824bf3b8b5d002889b0ecb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.592293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1201/2360] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:36 2022] DELETE /identity/v3/users/0fae5f3019824bf3b8b5d002889b0ecb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:36.761948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:36.763474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:36.778673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70438c1a-af30-4b4c-b3ff-4e1df9ad4beb admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:36.836196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:36.836995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:36.837281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:36.837556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] PATH_INFO: `/v3/projects/a155379c55514a56a67597939b0f4630` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:36.838784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a155379c55514a56a67597939b0f4630)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:36.840735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:36.874182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:36.891198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a155379c55514a56a67597939b0f4630', 'request_id': 'req-da1896ee-7660-4f10-b60f-8933294dcbc4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.898206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a155379c55514a56a67597939b0f4630', 'request_id': 'req-da1896ee-7660-4f10-b60f-8933294dcbc4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.903781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a155379c55514a56a67597939b0f4630', 'request_id': 'req-da1896ee-7660-4f10-b60f-8933294dcbc4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.916703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] The token cache is being invalidate because project a155379c55514a56a67597939b0f4630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:36.917082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.917783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-da1896ee-7660-4f10-b60f-8933294dcbc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:36.919526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1160/2361] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:36 2022] DELETE /identity/v3/projects/a155379c55514a56a67597939b0f4630 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:36.924543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f650b-e391-438d-aa31-8625108469c7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:36.924826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f650b-e391-438d-aa31-8625108469c7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:36.925110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f650b-e391-438d-aa31-8625108469c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:36.942582 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-014f650b-e391-438d-aa31-8625108469c7 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:36.943854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-014f650b-e391-438d-aa31-8625108469c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:37.003823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1202/2362] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:37.012427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.012719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.012999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.032022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:37.033350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:37.041409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1161/2363] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:13:37.045474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f650b-e391-438d-aa31-8625108469c7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.055070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f650b-e391-438d-aa31-8625108469c7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.059534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.060177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.060458 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.060732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.061450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:13:37.061753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.063372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.075627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1203/2364] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:13:37.079700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.089279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bea42-f986-4911-b0a4-8c2d9203ec2c None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.093754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.094423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.094734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.095011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.095698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.097566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.122269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1162/2365] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:37.127004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.137660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4329820-3fb5-4b22-b156-6a1a6c88aea7 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.167140 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:37.197376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.198039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.198318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.198587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.199302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.201209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.221366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1204/2366] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:37.225413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.235549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c4ae-1e47-4022-b747-930a80353e40 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.240085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.240738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.241019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.241796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.242518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.244213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.255260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1163/2367] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:37.259861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.269845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8cd4a-fd67-45a7-9f64-7e59b4cd9fc0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.274630 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630a18-7042-4b98-bd8e-e872dc6cf1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.275298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630a18-7042-4b98-bd8e-e872dc6cf1da admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.275584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630a18-7042-4b98-bd8e-e872dc6cf1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.275861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630a18-7042-4b98-bd8e-e872dc6cf1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.276591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630a18-7042-4b98-bd8e-e872dc6cf1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.278186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630a18-7042-4b98-bd8e-e872dc6cf1da admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.288232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1205/2368] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:13:37.292275 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.302038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5657ed-23c9-4aca-8af3-ae7646a02533 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.307098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.307969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.308247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.308547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] PATH_INFO: `/v3/projects/c9271e2548e8415badd05aac1bcda58d/users/47fc1312757946488de925b72957ab86/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.324157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9271e2548e8415badd05aac1bcda58d, user_id=47fc1312757946488de925b72957ab86, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.326823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.339770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1164/2369] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:37 2022] PUT /identity/v3/projects/c9271e2548e8415badd05aac1bcda58d/users/47fc1312757946488de925b72957ab86/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:37.349697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.349989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.350279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.376421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] MFA Rules not processed for user `47fc1312757946488de925b72957ab86`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:37.377608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:37.439247 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1206/2370] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:37.444033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.504359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae009b3-9e90-4a04-95b4-6c58d9b639cf admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.512019 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.512672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.512954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.513224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.513903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.515849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.539942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1165/2371] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:13:37.543940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.555098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae6439-28b3-4666-9bfe-dc55e5d018cb None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.610540 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.611232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.611511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.611788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.612499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.614732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.634442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1207/2372] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:37.638379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.648754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5b81f-3095-4c86-94e8-76214c0e9c9b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.654724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.655400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.655716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.656000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.656733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.658374 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.669544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1166/2373] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:37.673720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.684135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c3885-4a83-4cad-b600-dc34e8087b50 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.688761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.689635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.689923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.690203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] PATH_INFO: `/v3/projects/73bc6709919743a48149c456fdd93dc3/users/91e3a48741e0460bad4195220335e394/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.704649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bc6709919743a48149c456fdd93dc3, user_id=91e3a48741e0460bad4195220335e394, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.707361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.720062 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1208/2374] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:37 2022] PUT /identity/v3/projects/73bc6709919743a48149c456fdd93dc3/users/91e3a48741e0460bad4195220335e394/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:37.723955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.734078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b869c18-b4cb-4cd7-8458-5e21f0932443 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.759572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:37.796536 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63b585-e23b-48c6-9dc4-7b56637497d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.797241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b585-e23b-48c6-9dc4-7b56637497d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.797522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b585-e23b-48c6-9dc4-7b56637497d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.797793 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b585-e23b-48c6-9dc4-7b56637497d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.798531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63b585-e23b-48c6-9dc4-7b56637497d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.800181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63b585-e23b-48c6-9dc4-7b56637497d7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.811139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1167/2375] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:37.815118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.825681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511047-fa31-49ba-8146-96aa9b598766 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:37.830305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:37.831240 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.831523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.831798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] PATH_INFO: `/v3/projects/73bc6709919743a48149c456fdd93dc3/users/91e3a48741e0460bad4195220335e394/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.833769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] RBAC: Authorizing `identity:create_grant(project_id=73bc6709919743a48149c456fdd93dc3, user_id=91e3a48741e0460bad4195220335e394, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:37.836476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:37.838826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:37.849538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1209/2376] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:37 2022] PUT /identity/v3/projects/73bc6709919743a48149c456fdd93dc3/users/91e3a48741e0460bad4195220335e394/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:37.861621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:37.861843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:37.862125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:37.891664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] MFA Rules not processed for user `91e3a48741e0460bad4195220335e394`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:37.892897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:37.964694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1168/2377] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:13:37.975581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:37.977992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:38.064100 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ad8d9-1f19-410e-8a72-3c3f8266238e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:38.068929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b7791-4c31-4a42-960a-76bb94507efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:38.071064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7791-4c31-4a42-960a-76bb94507efc service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:38.071928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7791-4c31-4a42-960a-76bb94507efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:38.072393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b7791-4c31-4a42-960a-76bb94507efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:38.090238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b7791-4c31-4a42-960a-76bb94507efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:38.092359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b7791-4c31-4a42-960a-76bb94507efc service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:38.107200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1210/2378] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:38.464949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:38.525583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06ca60-1a6d-4f0e-9ceb-d5b69b71d261 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:38.530124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:38.531020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:38.531303 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:38.531576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] PATH_INFO: `/v3/users/47fc1312757946488de925b72957ab86` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:38.532751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47fc1312757946488de925b72957ab86)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:38.534998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:38.578372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47fc1312757946488de925b72957ab86'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:38.582009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:38.586750 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47fc1312757946488de925b72957ab86'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:38.600372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47fc1312757946488de925b72957ab86'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:38.601014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47fc1312757946488de925b72957ab86'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:38.601635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47fc1312757946488de925b72957ab86'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:38.613635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1169/2379] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:38 2022] DELETE /identity/v3/users/47fc1312757946488de925b72957ab86 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:38.618166 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:38.618396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:38.618729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:38.635966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:38.637187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:38.696437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1211/2380] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:38.705986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:38.707431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:38.763945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebfeca-4bcd-48dd-a631-6dd07b778932 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:38.767562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:38.768205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:38.768488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:38.768767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:38.777829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:38.779304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:38.788761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1170/2381] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:39.053218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:39.055178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:39.067296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515153e-1058-46d4-bd44-a20439a8a759 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:39.072519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:39.073392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.073678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.073954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] PATH_INFO: `/v3/projects/c9271e2548e8415badd05aac1bcda58d` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.075159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9271e2548e8415badd05aac1bcda58d)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:39.077060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:39.125236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9271e2548e8415badd05aac1bcda58d', 'request_id': 'req-47c087db-c5d7-4631-98a6-dacb5e1e6d30'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.131580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9271e2548e8415badd05aac1bcda58d', 'request_id': 'req-47c087db-c5d7-4631-98a6-dacb5e1e6d30'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.137291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9271e2548e8415badd05aac1bcda58d', 'request_id': 'req-47c087db-c5d7-4631-98a6-dacb5e1e6d30'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.146167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:39.150634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] The token cache is being invalidate because project c9271e2548e8415badd05aac1bcda58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:39.151044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.151859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.153907 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1212/2382] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:39 2022] DELETE /identity/v3/projects/c9271e2548e8415badd05aac1bcda58d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:39.157927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:39.168414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e5f0f-f6bc-46df-a070-dcbfe3671a10 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:39.223388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:39.224251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.224531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.224804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] PATH_INFO: `/v3/users/91e3a48741e0460bad4195220335e394` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.226476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e3a48741e0460bad4195220335e394)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:39.228435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:39.280497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e3a48741e0460bad4195220335e394'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.287367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e3a48741e0460bad4195220335e394'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.301279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e3a48741e0460bad4195220335e394'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.301961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e3a48741e0460bad4195220335e394'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.302575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-38bc1f39-4537-42f2-910e-f757e122b9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e3a48741e0460bad4195220335e394'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.305289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:39.318234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1171/2383] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:39 2022] DELETE /identity/v3/users/91e3a48741e0460bad4195220335e394 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:39.586236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:39.588097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:39.603128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c087db-c5d7-4631-98a6-dacb5e1e6d30 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:39.660387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:39.661291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.661571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.661855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] PATH_INFO: `/v3/projects/73bc6709919743a48149c456fdd93dc3` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.663640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73bc6709919743a48149c456fdd93dc3)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:39.665533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:39.709327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73bc6709919743a48149c456fdd93dc3', 'request_id': 'req-444997c1-cffd-4d6e-82aa-c57933062998'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.715820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73bc6709919743a48149c456fdd93dc3', 'request_id': 'req-444997c1-cffd-4d6e-82aa-c57933062998'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.721178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73bc6709919743a48149c456fdd93dc3', 'request_id': 'req-444997c1-cffd-4d6e-82aa-c57933062998'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.733353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] The token cache is being invalidate because project 73bc6709919743a48149c456fdd93dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:39.733634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.734241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-444997c1-cffd-4d6e-82aa-c57933062998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:39.735771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1213/2384] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:39 2022] DELETE /identity/v3/projects/73bc6709919743a48149c456fdd93dc3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:39.740656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.741169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.741302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.759004 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:39.760181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:39.820689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1172/2385] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:39.828758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.829048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.829357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.847869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:39.849065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:39.856786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1214/2386] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:39.860925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:39.870448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd98cd3-48af-4a6f-b219-70c2bbe2c184 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:39.875015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:39.875667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.875957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.876233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.876961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:13:39.877288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:39.878916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:39.891202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1173/2387] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:13:39.895309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:39.904999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9ac02-9835-4d02-90fc-ed0594ed267a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:39.909616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:39.910261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:39.910634 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:39.910854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:39.911549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:39.913428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:39.938155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1215/2388] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:39 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:39.942333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:39.952567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0d867-3515-449c-9e84-255a138399ad admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:40.133392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:40.134316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:40.134595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:40.134894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:40.135584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:40.137483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:40.157482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1174/2389] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:39 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:40.161628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:40.171946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de7dc9-4364-4719-a818-ed17376ddda0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:40.176957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:40.177605 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:40.177889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:40.178164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:40.178978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:40.180592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:40.191209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1216/2390] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:40.195907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:40.205689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5ec2e-5cb3-4078-8a9c-6b38a17a2004 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:40.210876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562a80-1925-46cd-87f1-2873c86bd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:40.211525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562a80-1925-46cd-87f1-2873c86bd79b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:40.211806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562a80-1925-46cd-87f1-2873c86bd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:40.212079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562a80-1925-46cd-87f1-2873c86bd79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:40.212828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562a80-1925-46cd-87f1-2873c86bd79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:40.214409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562a80-1925-46cd-87f1-2873c86bd79b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:40.224158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1175/2391] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:40.228013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:40.237834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc416a-3a0c-446f-a63d-e2fa86abe062 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:40.243144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:40.244019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:40.244306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:40.244585 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] PATH_INFO: `/v3/projects/7cf1315c9351478ea527a15677048d12/users/7bba1ff0b0aa484cb997143002341e3f/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:40.259649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf1315c9351478ea527a15677048d12, user_id=7bba1ff0b0aa484cb997143002341e3f, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:40.262339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:40.275593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1217/2392] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:40 2022] PUT /identity/v3/projects/7cf1315c9351478ea527a15677048d12/users/7bba1ff0b0aa484cb997143002341e3f/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:13:40.280562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae89660-4ce5-48a9-848c-19a9d317198b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:40.280850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae89660-4ce5-48a9-848c-19a9d317198b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:40.281124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae89660-4ce5-48a9-848c-19a9d317198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:40.298284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-3ae89660-4ce5-48a9-848c-19a9d317198b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:40.299498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3ae89660-4ce5-48a9-848c-19a9d317198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:40.347592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:40.357580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1176/2393] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:13:40.368205 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:40.369858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:40.400313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:40.428044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bbaf8-bb40-4a4d-9523-9440cac4bff0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:40.432261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:40.432908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:40.433195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:40.433468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:40.442576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:40.444238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:40.454352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1218/2394] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:41.115101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.115586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.115766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.143511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] MFA Rules not processed for user `7bba1ff0b0aa484cb997143002341e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:41.144720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:41.219166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1177/2395] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:41.223640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.235644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986edcf-a647-4a4e-af2c-a955c4beee34 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.240919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.242093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.242378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.242683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.243375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.245267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.281176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1219/2396] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:41 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:13:41.285617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.296810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9a3b3-cec8-44e7-9691-84e6dc0c9c15 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.353802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.354469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.354782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.355057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.355739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.357655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.382014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1178/2397] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:41 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:41.383633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.394465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c9ff2-b192-4c15-b9c7-2059cfe55b3c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.399915 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2efb01-f3eb-4ea4-bac0-380e9b7762e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.400928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2efb01-f3eb-4ea4-bac0-380e9b7762e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.401317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2efb01-f3eb-4ea4-bac0-380e9b7762e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.402061 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2efb01-f3eb-4ea4-bac0-380e9b7762e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.403315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2efb01-f3eb-4ea4-bac0-380e9b7762e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.406145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2efb01-f3eb-4ea4-bac0-380e9b7762e6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.422187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1220/2398] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:41.423414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.435690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f96cf-424b-4358-9946-b0cfd7aca6ab admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.436985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.437275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.437557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.440302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.441181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.441469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.441743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] PATH_INFO: `/v3/projects/b84b29c4de454420b84220acd8c5103a/users/f4d843abd1ca493bbfd19056147207da/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.461723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84b29c4de454420b84220acd8c5103a, user_id=f4d843abd1ca493bbfd19056147207da, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.464429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.466544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:41.467665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:41.480214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1179/2399] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:41 2022] PUT /identity/v3/projects/b84b29c4de454420b84220acd8c5103a/users/f4d843abd1ca493bbfd19056147207da/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:41.484231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.528750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1221/2400] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:13:41.534809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:41.547917 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac53f7-a854-4889-8935-0b1930563eac admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.552785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.553507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.554380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.554755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.555571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.557065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.567965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1180/2401] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:13:41.572197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.580131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:41.581859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:41.583684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68232350-65e0-4136-850b-fb93afe70d47 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.588381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716c881-25c1-4523-801b-cb9e11b3493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.589267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716c881-25c1-4523-801b-cb9e11b3493a admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.589552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716c881-25c1-4523-801b-cb9e11b3493a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.589858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716c881-25c1-4523-801b-cb9e11b3493a admin admin] PATH_INFO: `/v3/projects/b84b29c4de454420b84220acd8c5103a/users/f4d843abd1ca493bbfd19056147207da/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.591859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716c881-25c1-4523-801b-cb9e11b3493a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84b29c4de454420b84220acd8c5103a, user_id=f4d843abd1ca493bbfd19056147207da, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.594503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716c881-25c1-4523-801b-cb9e11b3493a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.613331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1222/2402] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:41 2022] PUT /identity/v3/projects/b84b29c4de454420b84220acd8c5103a/users/f4d843abd1ca493bbfd19056147207da/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:41.660892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488138-97db-4a04-b3b6-78507b5a460f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:41.735478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:41.736237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:41.736529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:41.736810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:41.797396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:41.799134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:41.808842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1181/2403] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:13:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 21:13:42.483503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:42.484037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:42.484370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:42.516067 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] MFA Rules not processed for user `f4d843abd1ca493bbfd19056147207da`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:42.518220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:42.598754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1223/2404] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:42.612254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:42.613717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:42.657416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:42.681618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04309963-80b3-453d-a239-ef152fd73e57 service swift] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:42.685376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fd2e7-df2c-4b0f-bb54-51c08892a30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:42.686091 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fd2e7-df2c-4b0f-bb54-51c08892a30a service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:42.686442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fd2e7-df2c-4b0f-bb54-51c08892a30a service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:42.686818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fd2e7-df2c-4b0f-bb54-51c08892a30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:42.748389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fd2e7-df2c-4b0f-bb54-51c08892a30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:42.749839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fd2e7-df2c-4b0f-bb54-51c08892a30a service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:42.760421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1182/2405] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:44.269304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:44.271068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:44.284501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:44.290425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710b9e4-d19a-4b07-bcc3-112d43ddc83d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:44.297015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b05cf-2905-4200-8385-a486329c2735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:44.297742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b05cf-2905-4200-8385-a486329c2735 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:44.298095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b05cf-2905-4200-8385-a486329c2735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:44.298438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b05cf-2905-4200-8385-a486329c2735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:44.309864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b05cf-2905-4200-8385-a486329c2735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:44.311461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b05cf-2905-4200-8385-a486329c2735 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:44.323201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1224/2406] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:52.755137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=33 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:54.317819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:55.954882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591705ee-7715-4525-9823-0b0627c3907e None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:55.955437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591705ee-7715-4525-9823-0b0627c3907e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:55.955437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591705ee-7715-4525-9823-0b0627c3907e None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:55.956942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1183/2407] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:13:55.963903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def292-53fd-4527-8f33-a81dc9f6a5cb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:55.964352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def292-53fd-4527-8f33-a81dc9f6a5cb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:55.964489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def292-53fd-4527-8f33-a81dc9f6a5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:55.983672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-49def292-53fd-4527-8f33-a81dc9f6a5cb None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:55.984988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-49def292-53fd-4527-8f33-a81dc9f6a5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:56.055694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1225/2408] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:56.067415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591705ee-7715-4525-9823-0b0627c3907e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:56.069879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-591705ee-7715-4525-9823-0b0627c3907e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:56.133895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591705ee-7715-4525-9823-0b0627c3907e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:56.137427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8baea-562d-498e-bafc-adb2b6352083 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:56.138067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8baea-562d-498e-bafc-adb2b6352083 service glance] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.138346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8baea-562d-498e-bafc-adb2b6352083 service glance] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.138639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8baea-562d-498e-bafc-adb2b6352083 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.150780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8baea-562d-498e-bafc-adb2b6352083 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:56.152228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8baea-562d-498e-bafc-adb2b6352083 service glance] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:56.162736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1184/2409] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:56.228169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d304c1d-418e-474d-a943-d533b7ca6058 None None] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.228451 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d304c1d-418e-474d-a943-d533b7ca6058 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.228723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d304c1d-418e-474d-a943-d533b7ca6058 None None] PATH_INFO: `/` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.230127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1226/2410] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:13:56.234680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b3892-d958-446a-9990-5ef76e836cae None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.234971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b3892-d958-446a-9990-5ef76e836cae None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.235249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b3892-d958-446a-9990-5ef76e836cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.253696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b90b3892-d958-446a-9990-5ef76e836cae None None] MFA Rules not processed for user `df6ccbd929724922bdbb1150d543bacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:56.254947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b90b3892-d958-446a-9990-5ef76e836cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:56.306555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1185/2411] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:56.318015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d304c1d-418e-474d-a943-d533b7ca6058 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:56.329659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d304c1d-418e-474d-a943-d533b7ca6058 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:56.333281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608fe02-10c3-42ca-8eb1-8a82aadfed2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:56.333920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608fe02-10c3-42ca-8eb1-8a82aadfed2d service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.334206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608fe02-10c3-42ca-8eb1-8a82aadfed2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.334483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608fe02-10c3-42ca-8eb1-8a82aadfed2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.342268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608fe02-10c3-42ca-8eb1-8a82aadfed2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:56.343803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608fe02-10c3-42ca-8eb1-8a82aadfed2d service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:56.354966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1227/2412] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:56.439979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b304826-5364-4fd5-9018-27ca0ac6b574 None None] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.440270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b304826-5364-4fd5-9018-27ca0ac6b574 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.440546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b304826-5364-4fd5-9018-27ca0ac6b574 None None] PATH_INFO: `/` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.441906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1186/2413] 10.222.0.39 () {54 vars in 921 bytes} [Wed Jul 13 21:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:13:56.447995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.448284 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.448559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.466448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] MFA Rules not processed for user `a2048d28ca3b42d8b8d82f4754e44433`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:56.467681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:56.478180 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1228/2414] 10.222.0.39 () {58 vars in 1018 bytes} [Wed Jul 13 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:56.488440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b304826-5364-4fd5-9018-27ca0ac6b574 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:56.490130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1b304826-5364-4fd5-9018-27ca0ac6b574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:56.503682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b304826-5364-4fd5-9018-27ca0ac6b574 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:56.507400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:56.508295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:56.508588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:56.509365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:56.520031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:56.521451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:56.532561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1187/2415] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:57.703816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:57.769970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ec9de-a430-43bb-b510-3a558952d4df None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:57.775580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:57.776460 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:57.776761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:57.777019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] PATH_INFO: `/v3/users/7bba1ff0b0aa484cb997143002341e3f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:57.778501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bba1ff0b0aa484cb997143002341e3f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:57.780598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:57.829584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bba1ff0b0aa484cb997143002341e3f'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:57.838815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bba1ff0b0aa484cb997143002341e3f'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:57.850823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bba1ff0b0aa484cb997143002341e3f'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:57.851570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bba1ff0b0aa484cb997143002341e3f'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:57.852250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bba1ff0b0aa484cb997143002341e3f'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:57.854349 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:57.867358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1229/2416] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:57 2022] DELETE /identity/v3/users/7bba1ff0b0aa484cb997143002341e3f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:58.045576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:58.061243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcdb1ca-8158-4c45-aaad-946d22d79b19 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:58.122952 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:58.123821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.124107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.124384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] PATH_INFO: `/v3/projects/7cf1315c9351478ea527a15677048d12` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.125566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf1315c9351478ea527a15677048d12)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:58.127510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:58.169281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:58.177448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf1315c9351478ea527a15677048d12', 'request_id': 'req-3e607f19-9e46-43f2-be84-383aaf0b0633'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.184602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf1315c9351478ea527a15677048d12', 'request_id': 'req-3e607f19-9e46-43f2-be84-383aaf0b0633'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.190193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf1315c9351478ea527a15677048d12', 'request_id': 'req-3e607f19-9e46-43f2-be84-383aaf0b0633'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.202883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] The token cache is being invalidate because project 7cf1315c9351478ea527a15677048d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:58.203173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.203807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e607f19-9e46-43f2-be84-383aaf0b0633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.205397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1188/2417] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:58 2022] DELETE /identity/v3/projects/7cf1315c9351478ea527a15677048d12 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:13:58.209591 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:58.210797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.224338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59c12f-ba0d-4ee5-94a4-dd824737718d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:58.279482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b206-dc54-469a-872f-266696f97245 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.279857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b206-dc54-469a-872f-266696f97245 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.280146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638b206-dc54-469a-872f-266696f97245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.291628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:58.292589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.292883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.293169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] PATH_INFO: `/v3/users/f4d843abd1ca493bbfd19056147207da` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.294897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4d843abd1ca493bbfd19056147207da)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:58.297169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:58.299302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a638b206-dc54-469a-872f-266696f97245 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:58.300512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a638b206-dc54-469a-872f-266696f97245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.358120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4d843abd1ca493bbfd19056147207da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.365454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1189/2418] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:13:58.368271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4d843abd1ca493bbfd19056147207da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.381106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b206-dc54-469a-872f-266696f97245 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:58.383747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a638b206-dc54-469a-872f-266696f97245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.385219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4d843abd1ca493bbfd19056147207da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.386035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4d843abd1ca493bbfd19056147207da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.386846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4d843abd1ca493bbfd19056147207da'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.400610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1230/2419] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:58 2022] DELETE /identity/v3/users/f4d843abd1ca493bbfd19056147207da => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:58.462884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638b206-dc54-469a-872f-266696f97245 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:58.524254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:58.524993 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.525344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.525690 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.527684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.545733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:58.576947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:58.578717 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.595732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:58.597330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1d922-1b8c-425d-afcf-60260058e827 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:58.610800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1190/2420] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 21:13:58.653389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965de8d-fbef-43e5-9e4e-9f81cbc590f3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:58.657949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:58.658838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.659120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.659554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] PATH_INFO: `/v3/projects/b84b29c4de454420b84220acd8c5103a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.661167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84b29c4de454420b84220acd8c5103a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:58.662498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:58.675914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:58.732202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84b29c4de454420b84220acd8c5103a', 'request_id': 'req-2ad0afc2-50c1-4a00-b813-61da65bc3c93'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.741696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84b29c4de454420b84220acd8c5103a', 'request_id': 'req-2ad0afc2-50c1-4a00-b813-61da65bc3c93'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.748698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84b29c4de454420b84220acd8c5103a', 'request_id': 'req-2ad0afc2-50c1-4a00-b813-61da65bc3c93'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.771159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] The token cache is being invalidate because project b84b29c4de454420b84220acd8c5103a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:13:58.771827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.772858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2ad0afc2-50c1-4a00-b813-61da65bc3c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:58.775047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1231/2421] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:13:58 2022] DELETE /identity/v3/projects/b84b29c4de454420b84220acd8c5103a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:13:58.782545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.783454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.784338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.822143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:58.824122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.918297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1191/2422] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:58.926087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.926382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.926693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.953577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:58.954876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:58.964780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1232/2423] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:58.971154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:58.983814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cc3b5-7d9a-4498-b0b1-10150302da72 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:58.989539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:58.990218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:58.990500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:58.990841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:58.991581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:13:58.991887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:58.993499 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.007537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1192/2424] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:13:59.011921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.021587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff83e9-a36f-4825-9a21-84918b776d71 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.027010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.027673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.027966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.028242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.028942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.030868 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.060826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1233/2425] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:13:59.062032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.072341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1647b0-ad02-450e-a47f-c4b2073a71a2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.135533 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.136379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.136651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.137059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.137771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.140260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.164708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1193/2426] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:13:59 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:13:59.165941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.177194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765be93f-fe01-4e6b-85f9-c86346ce4839 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.183065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.183928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.184210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.184643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.185989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.191066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.202672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1234/2427] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:13:59.206454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.219183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f716a8-d606-4b87-9d17-b708c7728b66 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.224597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630148c5-65ce-4014-bea3-1a152bc9761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.226129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630148c5-65ce-4014-bea3-1a152bc9761c admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.226437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630148c5-65ce-4014-bea3-1a152bc9761c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.226767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630148c5-65ce-4014-bea3-1a152bc9761c admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.227619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630148c5-65ce-4014-bea3-1a152bc9761c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.229227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630148c5-65ce-4014-bea3-1a152bc9761c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.239679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1194/2428] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:13:59.243919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.254566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27eef0-98c8-4c0e-a72d-a5276d673813 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.259920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.260877 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.261159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.261439 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] PATH_INFO: `/v3/projects/57cbc2dd1825472da33327934c237490/users/8270d9ac3928493bb998d5b1928de433/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.283604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57cbc2dd1825472da33327934c237490, user_id=8270d9ac3928493bb998d5b1928de433, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.286379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.306150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1235/2429] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:13:59 2022] PUT /identity/v3/projects/57cbc2dd1825472da33327934c237490/users/8270d9ac3928493bb998d5b1928de433/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:59.317399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.317705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.317958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.346687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] MFA Rules not processed for user `8270d9ac3928493bb998d5b1928de433`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:59.347908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:59.360040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:59.419122 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1195/2430] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:59.430902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.432598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:59.500261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a627017-bc78-40ab-877f-d3d1efff1473 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.504420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58d81a-cb97-4a9c-aede-093342c277a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.505157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58d81a-cb97-4a9c-aede-093342c277a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.505446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58d81a-cb97-4a9c-aede-093342c277a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.505726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58d81a-cb97-4a9c-aede-093342c277a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.517165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58d81a-cb97-4a9c-aede-093342c277a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.518997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58d81a-cb97-4a9c-aede-093342c277a9 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.529753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1236/2431] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:13:59.558704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.628984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4764b32-0c06-449f-b49d-b2d7d1681ab2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.635734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.636650 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.636930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.637199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] PATH_INFO: `/v3/users/8270d9ac3928493bb998d5b1928de433` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.638668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8270d9ac3928493bb998d5b1928de433)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.640400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.708168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8270d9ac3928493bb998d5b1928de433'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:59.716799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8270d9ac3928493bb998d5b1928de433'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:59.728819 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8270d9ac3928493bb998d5b1928de433'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:59.729468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8270d9ac3928493bb998d5b1928de433'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:59.730061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8270d9ac3928493bb998d5b1928de433'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:13:59.742797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1196/2432] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:13:59 2022] DELETE /identity/v3/users/8270d9ac3928493bb998d5b1928de433 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:13:59.747309 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.747768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.747862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.765761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:13:59.766974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:59.799603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:59.826216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1237/2433] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:13:59.836602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:13:59.838042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:13:59.872807 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:13:59.897083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee82157-f1a0-44d1-9f4f-fe4340554d84 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:13:59.901662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4ef96-a207-43aa-8e82-09d35adebd07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:13:59.902317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4ef96-a207-43aa-8e82-09d35adebd07 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:13:59.902901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4ef96-a207-43aa-8e82-09d35adebd07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:13:59.903188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4ef96-a207-43aa-8e82-09d35adebd07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:13:59.912680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4ef96-a207-43aa-8e82-09d35adebd07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:13:59.914124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4ef96-a207-43aa-8e82-09d35adebd07 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:13:59.924468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1197/2434] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:14:00.214351 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.216322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:00.242288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee435-e380-453a-ab89-4c9022a86c65 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.249011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.249902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.250184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.250466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] PATH_INFO: `/v3/projects/57cbc2dd1825472da33327934c237490` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.251748 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57cbc2dd1825472da33327934c237490)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.253668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.321290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57cbc2dd1825472da33327934c237490', 'request_id': 'req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:00.327856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57cbc2dd1825472da33327934c237490', 'request_id': 'req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:00.333923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57cbc2dd1825472da33327934c237490', 'request_id': 'req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:00.348168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] The token cache is being invalidate because project 57cbc2dd1825472da33327934c237490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:14:00.348467 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:00.349088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cc42a4f-96ef-4eb1-8487-f42f97c12e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:00.351489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1238/2435] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:14:00 2022] DELETE /identity/v3/projects/57cbc2dd1825472da33327934c237490 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:14:00.356646 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.357142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.357223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.376608 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:00.377831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:00.442783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1198/2436] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:00.450561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.450923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.451208 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.471045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:00.472279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:00.481760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1239/2437] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:00.485815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.495794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3f4b-3dd5-444e-a669-e8bfbd45abfa None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.500575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.501239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.501581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.501795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.502713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:14:00.503716 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.505318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.517959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1199/2438] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:14:00.522817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.532775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0ce39-f354-45a6-b899-215bd4665886 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.537545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.538811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.539104 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.539386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.540085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.542029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.574292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1240/2439] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:14:00 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:14:00.578377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.589996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e9406-f84f-4504-a604-f0141d0caa32 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.650375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.651247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.651604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.651956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.652753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.654778 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.676823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1200/2440] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:14:00 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:14:00.680594 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.691397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3475e-5d1d-4eb0-861b-63cb55f00a99 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.696957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.697771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.698135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.698487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.699350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.701089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.713281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1241/2441] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:14:00.717246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.729121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7326dde-2e30-4310-ac1d-1553be8a88ba admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.734813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f7d56-7f4f-4abf-b901-3a96fb81c818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.735572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f7d56-7f4f-4abf-b901-3a96fb81c818 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.735932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f7d56-7f4f-4abf-b901-3a96fb81c818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.736294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f7d56-7f4f-4abf-b901-3a96fb81c818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.737109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f7d56-7f4f-4abf-b901-3a96fb81c818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.738754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f7d56-7f4f-4abf-b901-3a96fb81c818 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.760240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1201/2442] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:14:00.764371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.777917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4313440-97e2-4f9d-bfee-7933aa8c1bdf admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:00.784320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:00.784996 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.785468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.785754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] PATH_INFO: `/v3/projects/7d083c2a9d6d4a72a299b78e56e00db3/users/110a674bd0d4424886a4ba1f52007c7e/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.804213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d083c2a9d6d4a72a299b78e56e00db3, user_id=110a674bd0d4424886a4ba1f52007c7e, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:00.806964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:00.826709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1242/2443] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:14:00 2022] PUT /identity/v3/projects/7d083c2a9d6d4a72a299b78e56e00db3/users/110a674bd0d4424886a4ba1f52007c7e/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:14:00.836774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:00.837885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:00.837885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:00.875507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] MFA Rules not processed for user `110a674bd0d4424886a4ba1f52007c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:00.876762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:00.918518 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:00.956285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1202/2444] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:00.968392 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:00.970109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:01.031774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bea09f-c63c-42a1-bb2d-b5b9110681a9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:01.035429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e999f35-28e9-4147-ab4c-7ae6d13cf9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:01.036084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e999f35-28e9-4147-ab4c-7ae6d13cf9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:01.036372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e999f35-28e9-4147-ab4c-7ae6d13cf9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:01.036652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e999f35-28e9-4147-ab4c-7ae6d13cf9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:01.046253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e999f35-28e9-4147-ab4c-7ae6d13cf9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:01.047743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e999f35-28e9-4147-ab4c-7ae6d13cf9d5 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:01.057223 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1243/2445] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:14:03.688156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:03.750798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b45b3-d66b-4878-834c-78542c2b3628 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:03.755677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:03.757216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:03.757493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:03.758028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] PATH_INFO: `/v3/users/110a674bd0d4424886a4ba1f52007c7e` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:03.759260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110a674bd0d4424886a4ba1f52007c7e)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:03.761187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:03.809570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110a674bd0d4424886a4ba1f52007c7e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:03.817069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110a674bd0d4424886a4ba1f52007c7e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:03.830432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110a674bd0d4424886a4ba1f52007c7e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:03.831257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110a674bd0d4424886a4ba1f52007c7e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:03.831878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110a674bd0d4424886a4ba1f52007c7e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:03.845241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1203/2446] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:14:03 2022] DELETE /identity/v3/users/110a674bd0d4424886a4ba1f52007c7e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:14:03.849749 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:03.850790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:03.850902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:03.856525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:03.870590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:03.871959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:03.935380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1244/2447] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:03.945135 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:03.946686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:04.006222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea3136-6fa3-43f7-bc77-82e7e4c07e87 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.010310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05781c-f067-459d-a24d-6d6d10a607ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.010947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05781c-f067-459d-a24d-6d6d10a607ce service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.011226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05781c-f067-459d-a24d-6d6d10a607ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.011503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05781c-f067-459d-a24d-6d6d10a607ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.017886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:04.022779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05781c-f067-459d-a24d-6d6d10a607ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.024853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05781c-f067-459d-a24d-6d6d10a607ce service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.036219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1204/2448] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:14:04.341237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.343018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:04.355732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f73d6-e6cf-4351-9e6c-fda9109b749c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.361878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.362978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.363665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.363665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] PATH_INFO: `/v3/projects/7d083c2a9d6d4a72a299b78e56e00db3` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.365325 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d083c2a9d6d4a72a299b78e56e00db3)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.367700 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.429057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d083c2a9d6d4a72a299b78e56e00db3', 'request_id': 'req-912955a2-58e8-45f8-8c6e-5c9d35271016'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:04.436213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d083c2a9d6d4a72a299b78e56e00db3', 'request_id': 'req-912955a2-58e8-45f8-8c6e-5c9d35271016'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:04.442878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d083c2a9d6d4a72a299b78e56e00db3', 'request_id': 'req-912955a2-58e8-45f8-8c6e-5c9d35271016'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:04.462580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] The token cache is being invalidate because project 7d083c2a9d6d4a72a299b78e56e00db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:14:04.463201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:04.463976 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-912955a2-58e8-45f8-8c6e-5c9d35271016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:04.465949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1245/2449] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:14:04 2022] DELETE /identity/v3/projects/7d083c2a9d6d4a72a299b78e56e00db3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:14:04.470788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.471194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.471389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.497777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:04.499112 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:04.570473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1205/2450] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:14:04.579007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.579297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.579580 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.598362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:04.599620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:04.608964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1246/2451] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:04.612562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.623865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5fa55-e326-44df-af15-acf003c6a3de None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.628537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.629203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.629487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.629763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.630507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:14:04.630851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.633285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.645429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1206/2452] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:14:04.650331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.660789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea377b4e-595d-4a53-9be0-14e4f57bb1e3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.665840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.666587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.666975 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.667356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.668147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.670128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.699261 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1247/2453] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:14:04 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:14:04.700489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.711498 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3931f5-ad0d-451d-ae12-b681039cdbe0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.769259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.770116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.770507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.770900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.771818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.773841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.796654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1207/2454] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:14:04 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:14:04.800588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.813292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844179bf-871c-42e3-8fd1-19ea85ad0de5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.817985 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.819248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.819536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.819815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.820622 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.822162 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.833782 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1248/2455] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:14:04.838278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.848687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6f96c-b262-48ad-a710-e7d9fd259258 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.853927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43e5a1-3c0a-4f75-8448-882edb699cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.854591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43e5a1-3c0a-4f75-8448-882edb699cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.854904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43e5a1-3c0a-4f75-8448-882edb699cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.855186 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43e5a1-3c0a-4f75-8448-882edb699cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.856059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43e5a1-3c0a-4f75-8448-882edb699cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.857804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43e5a1-3c0a-4f75-8448-882edb699cfc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.869281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1208/2456] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:14:04.873472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:04.884633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563762da-b81c-4130-a0bb-1a0fd25f225f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:04.889664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:04.890562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.890898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.891177 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] PATH_INFO: `/v3/projects/899ef004c0094f22b3fa026e473d8311/users/07e05f421782441abadfb0482722f729/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.907529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=899ef004c0094f22b3fa026e473d8311, user_id=07e05f421782441abadfb0482722f729, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:04.910224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:04.925213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1249/2457] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:14:04 2022] PUT /identity/v3/projects/899ef004c0094f22b3fa026e473d8311/users/07e05f421782441abadfb0482722f729/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:14:04.936118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2f505-f8df-4f5e-b797-19c565dd9194 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:04.936407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2f505-f8df-4f5e-b797-19c565dd9194 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:04.936689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2f505-f8df-4f5e-b797-19c565dd9194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:04.974746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a8b2f505-f8df-4f5e-b797-19c565dd9194 None None] MFA Rules not processed for user `07e05f421782441abadfb0482722f729`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:04.976749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a8b2f505-f8df-4f5e-b797-19c565dd9194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:05.026170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:05.046587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1209/2458] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:05.059283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:05.061375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:05.088283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:05.122072 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51fc05-ced5-4e2d-b64b-7f9930edd6b1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:05.126675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:05.127436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:05.127858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:05.128199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:05.138545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:05.140049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:05.157719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1250/2459] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:14:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:14:15.043880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:15.146141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=16 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:20.697769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:20.698225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:20.698326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:20.716146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:20.717324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:20.771089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1210/2460] 10.222.0.39 () {58 vars in 967 bytes} [Wed Jul 13 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:20.782866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:20.784243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:20.853355 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9649-a3dd-41fa-93bb-554a5f48a6c7 service glance] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:20.858415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:20.859094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:20.859379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:20.859654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:20.867946 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:20.869651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:20.878390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1251/2461] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:14:30.764433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=25 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:30.875011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=32 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:47.486994 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:47.488400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:47.544982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e845f-2634-4571-9a12-1abbd67e0747 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:47.549657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:47.550317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:47.550607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:47.550910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:47.553219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:47.623144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:47.624649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc49490-c15f-4caf-af91-8c0d8d470635 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:47.637486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1211/2462] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:14:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:14:50.425882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:50.485633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dd586-ddb1-4e76-9842-13fd45cf5356 service swift] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:50.490076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:50.490954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:50.491252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:50.491524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] PATH_INFO: `/v3/users/07e05f421782441abadfb0482722f729` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:50.492673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07e05f421782441abadfb0482722f729)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:50.494570 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:50.539931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07e05f421782441abadfb0482722f729'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:50.547245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07e05f421782441abadfb0482722f729'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:50.558884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07e05f421782441abadfb0482722f729'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:50.559505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07e05f421782441abadfb0482722f729'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:50.560099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07e05f421782441abadfb0482722f729'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:50.571885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1252/2463] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:14:50 2022] DELETE /identity/v3/users/07e05f421782441abadfb0482722f729 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:14:50.576312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:50.577109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:50.577109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:50.594377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:14:50.595597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:50.652524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1212/2464] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:14:50.662565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:50.664104 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:50.719708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1df3-034e-4024-90a3-62773e900728 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:50.723323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:50.723956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:50.724237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:50.724507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:50.733974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:50.735410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:50.744685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1253/2465] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:14:51.006786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:51.008241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:51.020895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846796e1-862b-4beb-9059-19b1f8530c90 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:51.025718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:51.026339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:51.026681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:51.026949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] PATH_INFO: `/v3/projects/899ef004c0094f22b3fa026e473d8311` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:51.028189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=899ef004c0094f22b3fa026e473d8311)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:51.030980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:51.068497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:51.079247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899ef004c0094f22b3fa026e473d8311', 'request_id': 'req-55164375-0fd1-4b86-a80b-3565da8301a7'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:51.085359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899ef004c0094f22b3fa026e473d8311', 'request_id': 'req-55164375-0fd1-4b86-a80b-3565da8301a7'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:51.090815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899ef004c0094f22b3fa026e473d8311', 'request_id': 'req-55164375-0fd1-4b86-a80b-3565da8301a7'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:51.103282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] The token cache is being invalidate because project 899ef004c0094f22b3fa026e473d8311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:14:51.103567 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:51.104190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:14:51.105699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1213/2466] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:14:51 2022] DELETE /identity/v3/projects/899ef004c0094f22b3fa026e473d8311 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:14:57.044986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:14:57.053448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770cb40-9b19-4c52-be1f-708798e446a2 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:14:57.070984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:14:57.103577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:14:57.104219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:14:57.104504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:14:57.104779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:14:57.106548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:14:57.174086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:14:57.175609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:14:57.186397 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1254/2467] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:14:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:01.100404 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:03.940343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:03.941805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:03.953787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55164375-0fd1-4b86-a80b-3565da8301a7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:03.957445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:03.958080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:03.958365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:03.958666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:03.960497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:04.022470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:04.023922 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:04.033852 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1214/2468] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:06.884537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:06.886163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:06.900013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8224c-cdbc-4b35-a551-b5688dd0b5ad service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:06.904650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:06.905602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:06.905951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:06.906303 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] PATH_INFO: `/v3/users/fc4afeb23348438eabfe22f1a975f9ab` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:06.921041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc4afeb23348438eabfe22f1a975f9ab)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:06.922999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:06.968432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc4afeb23348438eabfe22f1a975f9ab'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:06.975636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc4afeb23348438eabfe22f1a975f9ab'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:06.987808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc4afeb23348438eabfe22f1a975f9ab'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:06.988431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc4afeb23348438eabfe22f1a975f9ab'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:06.989030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-4c6e7baa-c31c-420b-aef6-66f138d7ae76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc4afeb23348438eabfe22f1a975f9ab'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:07.001511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1255/2469] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:06 2022] DELETE /identity/v3/users/fc4afeb23348438eabfe22f1a975f9ab => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:07.172777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.183675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc547ae6-d7d8-426a-aa83-2316f25c16df service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.240701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.241553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.241834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.242109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] PATH_INFO: `/v3/projects/91bc9f9f4bc04202aabc75fc4bb9a6b2` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.252182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91bc9f9f4bc04202aabc75fc4bb9a6b2)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.254088 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.300699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91bc9f9f4bc04202aabc75fc4bb9a6b2', 'request_id': 'req-51e233ed-52e8-43dc-a6ac-11a51aaa888d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:07.307180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91bc9f9f4bc04202aabc75fc4bb9a6b2', 'request_id': 'req-51e233ed-52e8-43dc-a6ac-11a51aaa888d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:07.312483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91bc9f9f4bc04202aabc75fc4bb9a6b2', 'request_id': 'req-51e233ed-52e8-43dc-a6ac-11a51aaa888d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:07.323972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] The token cache is being invalidate because project 91bc9f9f4bc04202aabc75fc4bb9a6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:07.324258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:07.324886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-51e233ed-52e8-43dc-a6ac-11a51aaa888d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:07.326444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1215/2470] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:07 2022] DELETE /identity/v3/projects/91bc9f9f4bc04202aabc75fc4bb9a6b2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:07.332060 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379714e-a596-4918-8132-f16276a5c133 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.332559 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379714e-a596-4918-8132-f16276a5c133 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.332910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379714e-a596-4918-8132-f16276a5c133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.352371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4379714e-a596-4918-8132-f16276a5c133 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:07.353568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4379714e-a596-4918-8132-f16276a5c133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:07.372428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:07.414014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1256/2471] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:07.421804 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.422092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.422415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.440093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:07.441214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:07.448928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1216/2472] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:07.453481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379714e-a596-4918-8132-f16276a5c133 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.462911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379714e-a596-4918-8132-f16276a5c133 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.467978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.468620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.468901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.469174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.469888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:07.470191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.471805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.483152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1257/2473] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:07.487221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.496308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf25060-d075-41e0-a16d-0902712ef5c2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.501021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.501637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.501921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.502195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.502910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.505003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.509050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:07.543420 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1217/2474] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:07 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:07.547736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.558270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06eee1-270a-482f-a966-a7d691925768 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.615488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.616164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.616450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.616729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.617420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.619394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.641044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1258/2475] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:07 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:07.645196 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.655972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f859223-2a69-4dfa-b6f3-0f6904498eef admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.660915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.661573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.661862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.662143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.662910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.664505 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.674368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1218/2476] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:07.678187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.688150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b0dc5-6dda-4bee-8ce7-3f82267d57ea admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.692775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54976755-cb03-4815-9610-eb8ae55a4d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.693395 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54976755-cb03-4815-9610-eb8ae55a4d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.693680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54976755-cb03-4815-9610-eb8ae55a4d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.693958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54976755-cb03-4815-9610-eb8ae55a4d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.694713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54976755-cb03-4815-9610-eb8ae55a4d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.696301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54976755-cb03-4815-9610-eb8ae55a4d8f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.706558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1259/2477] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:07.710535 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.720899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb1683-86f7-4e26-8235-99a51ef5360a admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.725425 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.726290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.726569 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.726876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] PATH_INFO: `/v3/projects/162650c7841a4aa5aec347d954654eb1/users/b556e6b0f0664d71950559aa194558bb/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.742074 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162650c7841a4aa5aec347d954654eb1, user_id=b556e6b0f0664d71950559aa194558bb, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.744723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.758051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1219/2478] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:07 2022] PUT /identity/v3/projects/162650c7841a4aa5aec347d954654eb1/users/b556e6b0f0664d71950559aa194558bb/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:07.769858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.770200 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.770493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.796122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] MFA Rules not processed for user `b556e6b0f0664d71950559aa194558bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:07.797318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:07.859101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1260/2479] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:07.869592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:07.871310 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:07.927985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70564ee5-2d31-4877-ab06-7cf4699b5819 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:07.932021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b18221-03aa-47cc-844e-e003652d328b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:07.932672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b18221-03aa-47cc-844e-e003652d328b service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:07.932959 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b18221-03aa-47cc-844e-e003652d328b service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:07.933236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b18221-03aa-47cc-844e-e003652d328b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:07.942109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b18221-03aa-47cc-844e-e003652d328b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:07.943615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b18221-03aa-47cc-844e-e003652d328b service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:07.952957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1220/2480] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:16.191050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:16.211265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:16.254310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa6c38-ff01-4e2f-a1e3-815689436d3d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:16.259187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:16.260055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:16.260338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:16.260614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] PATH_INFO: `/v3/users/b556e6b0f0664d71950559aa194558bb` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:16.261800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b556e6b0f0664d71950559aa194558bb)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:16.263760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:16.312123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b556e6b0f0664d71950559aa194558bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.319410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b556e6b0f0664d71950559aa194558bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.331212 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b556e6b0f0664d71950559aa194558bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.331846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b556e6b0f0664d71950559aa194558bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.332481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b556e6b0f0664d71950559aa194558bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.345028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1261/2481] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:16 2022] DELETE /identity/v3/users/b556e6b0f0664d71950559aa194558bb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:16.349501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:16.349986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:16.350092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:16.367619 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:16.368839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:16.427212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1221/2482] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:16.437732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:16.439183 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:16.495483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5cdf2-24b7-482e-aa2c-aa3c1968e1fa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:16.499135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f46748-7a69-42da-bfb4-a87b69805587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:16.499792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f46748-7a69-42da-bfb4-a87b69805587 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:16.500077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f46748-7a69-42da-bfb4-a87b69805587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:16.500357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f46748-7a69-42da-bfb4-a87b69805587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:16.509556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f46748-7a69-42da-bfb4-a87b69805587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:16.511021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f46748-7a69-42da-bfb4-a87b69805587 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:16.520789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1262/2483] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:15:16.792915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:16.794379 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:16.807098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b210e-85d2-442a-bcc0-40e6134e3aa8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:16.812408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:16.813421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:16.813707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:16.813987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] PATH_INFO: `/v3/projects/162650c7841a4aa5aec347d954654eb1` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:16.815207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=162650c7841a4aa5aec347d954654eb1)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:16.817142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:16.858260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:16.867802 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162650c7841a4aa5aec347d954654eb1', 'request_id': 'req-42969940-d4b1-4b9e-aff5-478cd1b205dd'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.875312 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162650c7841a4aa5aec347d954654eb1', 'request_id': 'req-42969940-d4b1-4b9e-aff5-478cd1b205dd'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.881146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162650c7841a4aa5aec347d954654eb1', 'request_id': 'req-42969940-d4b1-4b9e-aff5-478cd1b205dd'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.895402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] The token cache is being invalidate because project 162650c7841a4aa5aec347d954654eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:16.895664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.896323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-42969940-d4b1-4b9e-aff5-478cd1b205dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:16.897884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1222/2484] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:16 2022] DELETE /identity/v3/projects/162650c7841a4aa5aec347d954654eb1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:16.903107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:16.903400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:16.903678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:16.921289 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:16.922484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:16.961329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:16.984931 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1263/2485] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:16.992562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:16.992859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:16.993169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.012225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:17.013605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:17.022200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1223/2486] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:17.026300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.036027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc6bfa-6774-4197-87cd-4fb03e83ef61 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.040641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.041294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.041588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.041869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.042597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:17.042937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.044537 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.056021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1264/2487] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:17.060359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.070304 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c0a8d-2624-44e1-84b2-7d9833c43137 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.075013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.075683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.075969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.076249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.076947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.079496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.104571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1224/2488] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:17 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:17.109156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.119390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446984d4-aeeb-43d9-990b-7288b9150aee admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.177260 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.177926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.178209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.178506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.179245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.181164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.201665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1265/2489] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:17 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:17.205875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.216163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92d5a-acd6-416e-83c7-6e08f1202ccc admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.220834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.221491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.221779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.222057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.222885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.224482 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.235887 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1225/2490] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:17.240122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.250609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4a0e-e2db-494d-9e9d-c845f9092a4f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.255357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046edc0-9ab2-4a3e-9657-58976b2e20df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.256010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046edc0-9ab2-4a3e-9657-58976b2e20df admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.256294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046edc0-9ab2-4a3e-9657-58976b2e20df admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.256571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046edc0-9ab2-4a3e-9657-58976b2e20df admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.257301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046edc0-9ab2-4a3e-9657-58976b2e20df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.259168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046edc0-9ab2-4a3e-9657-58976b2e20df admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.269093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1266/2491] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:17.273551 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.283465 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df959b-a249-43d1-a2ef-d5016a10707e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.288037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.288916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.289203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.289478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] PATH_INFO: `/v3/projects/83861bd2719840958a6a3c0eb7cbf54d/users/57b288426e844737830934372754ce4f/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.304414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83861bd2719840958a6a3c0eb7cbf54d, user_id=57b288426e844737830934372754ce4f, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.307115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.321570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1226/2492] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:17 2022] PUT /identity/v3/projects/83861bd2719840958a6a3c0eb7cbf54d/users/57b288426e844737830934372754ce4f/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:17.331859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546d05-5754-4f1b-b877-94096c15899e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.332149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546d05-5754-4f1b-b877-94096c15899e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.332428 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546d05-5754-4f1b-b877-94096c15899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.360176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-83546d05-5754-4f1b-b877-94096c15899e None None] MFA Rules not processed for user `57b288426e844737830934372754ce4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:17.361363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-83546d05-5754-4f1b-b877-94096c15899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:17.425003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1267/2493] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:17.429562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.491064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355d344-1fef-4cbf-a2aa-7e7bebbc3bf8 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.495810 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.496473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.496755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.497028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.497724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.500163 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.526573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1227/2494] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:17 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:17.530649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83546d05-5754-4f1b-b877-94096c15899e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.541030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83546d05-5754-4f1b-b877-94096c15899e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.573779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:17.596873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.597529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.597806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.598074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.598790 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.600692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.619096 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1268/2495] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:17 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:17.623160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.634256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a3e0a-c0b3-4120-b456-4d033a951a34 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.639174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.640476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.640768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.641116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.641905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.643835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.654587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1228/2496] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:17.658686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.668675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050dd1b1-e293-4233-9380-91f9d34f9854 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.673476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.674279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.674569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.674874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] PATH_INFO: `/v3/projects/53186ec5d92d477388f5630cb71d977b/users/2f60738c51a64defa72fd2f2c4ab7e4d/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.691081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=53186ec5d92d477388f5630cb71d977b, user_id=2f60738c51a64defa72fd2f2c4ab7e4d, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.693754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.707979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1269/2497] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:17 2022] PUT /identity/v3/projects/53186ec5d92d477388f5630cb71d977b/users/2f60738c51a64defa72fd2f2c4ab7e4d/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:17.711808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.721881 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f9e8f-c426-40c2-a8d6-5231ffdac63b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.758738 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:17.778940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db77954-033b-468f-8721-02abf8f40fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.779606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db77954-033b-468f-8721-02abf8f40fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.779892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db77954-033b-468f-8721-02abf8f40fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.780172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db77954-033b-468f-8721-02abf8f40fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.780905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db77954-033b-468f-8721-02abf8f40fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.782500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db77954-033b-468f-8721-02abf8f40fbb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.792333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1229/2498] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:17.796422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.806415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138cadc-b0e1-4513-9936-c7cffbbd01fe admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:17.811008 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:17.812372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.812653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.812924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] PATH_INFO: `/v3/projects/53186ec5d92d477388f5630cb71d977b/users/2f60738c51a64defa72fd2f2c4ab7e4d/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.814927 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53186ec5d92d477388f5630cb71d977b, user_id=2f60738c51a64defa72fd2f2c4ab7e4d, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:17.817775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:17.828616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1270/2499] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:17 2022] PUT /identity/v3/projects/53186ec5d92d477388f5630cb71d977b/users/2f60738c51a64defa72fd2f2c4ab7e4d/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:17.839446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:17.839741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:17.840027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:17.866141 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] MFA Rules not processed for user `2f60738c51a64defa72fd2f2c4ab7e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:17.867398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:17.935184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1230/2500] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:17.946965 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:17.948708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:18.004152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff6c2-d62f-4d0a-950b-c89f755456e6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:18.009473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc6a60-f89a-405c-8b41-0457b0196059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:18.010516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6a60-f89a-405c-8b41-0457b0196059 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:18.011065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6a60-f89a-405c-8b41-0457b0196059 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:18.011579 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc6a60-f89a-405c-8b41-0457b0196059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:18.023097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc6a60-f89a-405c-8b41-0457b0196059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:18.024514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc6a60-f89a-405c-8b41-0457b0196059 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:18.033816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1271/2501] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:18.398514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:18.457918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5aa3e-a125-4226-8e52-75b8f8053868 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:18.462398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:18.463277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:18.463560 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:18.463834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] PATH_INFO: `/v3/users/57b288426e844737830934372754ce4f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:18.464999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b288426e844737830934372754ce4f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:18.466920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:18.513845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b288426e844737830934372754ce4f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:18.520341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b288426e844737830934372754ce4f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:18.527209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:18.533539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b288426e844737830934372754ce4f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:18.534507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b288426e844737830934372754ce4f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:18.535181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b288426e844737830934372754ce4f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:18.548305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1231/2502] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:18 2022] DELETE /identity/v3/users/57b288426e844737830934372754ce4f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:18.552826 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:18.553290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:18.553386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:18.570499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:18.571789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:18.628998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1272/2503] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:18.638739 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:18.640115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:18.697696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8419-e755-4abb-9ebd-4bf37247e85d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:18.701382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:18.702029 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:18.702309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:18.702581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:18.712261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:18.713680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:18.723916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1232/2504] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:18.988226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:18.989688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:19.002889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d788ab-80e7-43e1-a8f2-4be024ecc309 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.007501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.008370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.008649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.009179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] PATH_INFO: `/v3/projects/83861bd2719840958a6a3c0eb7cbf54d` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.010366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83861bd2719840958a6a3c0eb7cbf54d)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.012327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.043339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:19.059374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83861bd2719840958a6a3c0eb7cbf54d', 'request_id': 'req-8cb6a897-8a1d-464e-acd6-937f4295cf00'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.065833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83861bd2719840958a6a3c0eb7cbf54d', 'request_id': 'req-8cb6a897-8a1d-464e-acd6-937f4295cf00'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.071250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83861bd2719840958a6a3c0eb7cbf54d', 'request_id': 'req-8cb6a897-8a1d-464e-acd6-937f4295cf00'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.083483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] The token cache is being invalidate because project 83861bd2719840958a6a3c0eb7cbf54d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:19.083804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.084459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.085963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1273/2505] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:18 2022] DELETE /identity/v3/projects/83861bd2719840958a6a3c0eb7cbf54d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:19.089874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.099496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ebcdb0-4295-4666-8668-df354a58431e service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.155814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.156683 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.156965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.157240 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] PATH_INFO: `/v3/users/2f60738c51a64defa72fd2f2c4ab7e4d` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.158402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f60738c51a64defa72fd2f2c4ab7e4d)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.160357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.205144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f60738c51a64defa72fd2f2c4ab7e4d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.212044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f60738c51a64defa72fd2f2c4ab7e4d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.223568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f60738c51a64defa72fd2f2c4ab7e4d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.224197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f60738c51a64defa72fd2f2c4ab7e4d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.224798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-2dc92a64-9291-40e3-96d9-dcc62ac6e7f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f60738c51a64defa72fd2f2c4ab7e4d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.236649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1233/2506] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:19 2022] DELETE /identity/v3/users/2f60738c51a64defa72fd2f2c4ab7e4d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:19.502385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.503849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:19.516780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a897-8a1d-464e-acd6-937f4295cf00 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.572265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.573135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.573423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.573703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] PATH_INFO: `/v3/projects/53186ec5d92d477388f5630cb71d977b` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.574962 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53186ec5d92d477388f5630cb71d977b)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.576917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.622297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53186ec5d92d477388f5630cb71d977b', 'request_id': 'req-90c68373-7e9c-4ac5-8ddb-388216030b49'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.628912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53186ec5d92d477388f5630cb71d977b', 'request_id': 'req-90c68373-7e9c-4ac5-8ddb-388216030b49'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.634432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53186ec5d92d477388f5630cb71d977b', 'request_id': 'req-90c68373-7e9c-4ac5-8ddb-388216030b49'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.646380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] The token cache is being invalidate because project 53186ec5d92d477388f5630cb71d977b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:19.646701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.647326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90c68373-7e9c-4ac5-8ddb-388216030b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:19.648840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1274/2507] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:19 2022] DELETE /identity/v3/projects/53186ec5d92d477388f5630cb71d977b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:19.654173 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.654662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.654782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.660991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:19.675097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:19.676328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:19.733679 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1234/2508] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:19.741661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.741952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.742233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.760145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:19.761326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:19.769383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1275/2509] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:19.773411 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.783680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d760d-e721-4c5b-b65c-9db2132e2bf6 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.788338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.788995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.789279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.789556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.790289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:19.790593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.792222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.808331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1235/2510] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:19.809348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.818514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16daed2-0990-4687-8a5b-76b7db48cf8a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.823134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.823785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.824068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.824343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.825035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.826952 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.852134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1276/2511] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:19 2022] POST /identity/v3/projects => generated 355 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:19.856143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.866781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7618a-5f5d-4412-a3ac-cb272912b97c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.921938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.923274 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.923563 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.923837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.924527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.926429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.946403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1236/2512] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:19 2022] POST /identity/v3/users => generated 278 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:19.950476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.960598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92807f2-ce27-4787-9bb2-021bbcccd781 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.965129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.965776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.966057 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:19.966330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:19.967089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:19.968664 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:19.979669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1277/2513] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:19.983713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:19.994063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928ab92-e4af-4ed1-a9f8-247411f24cd3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:19.998766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a9183-1e6a-4b4c-9538-256ec3be4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:19.999440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a9183-1e6a-4b4c-9538-256ec3be4c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:19.999725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a9183-1e6a-4b4c-9538-256ec3be4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.000003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a9183-1e6a-4b4c-9538-256ec3be4c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.000755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a9183-1e6a-4b4c-9538-256ec3be4c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.002345 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a9183-1e6a-4b4c-9538-256ec3be4c4b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.012302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1237/2514] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:20.016153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.026158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33465da-9690-44ed-ae2c-29b838b4527d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.030783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.031648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.032443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.032716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] PATH_INFO: `/v3/projects/e4b567c3dd80469e8d7302f2f551aef3/users/5384f8e641fd4837a0d1d56584f8559a/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.034989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:20.050925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b567c3dd80469e8d7302f2f551aef3, user_id=5384f8e641fd4837a0d1d56584f8559a, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.053649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.065853 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1278/2515] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:20 2022] PUT /identity/v3/projects/e4b567c3dd80469e8d7302f2f551aef3/users/5384f8e641fd4837a0d1d56584f8559a/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:20.075851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.076144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.076436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.103092 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] MFA Rules not processed for user `5384f8e641fd4837a0d1d56584f8559a`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:20.104296 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:20.163582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:20.169337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1238/2516] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:20.173953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.234553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dab8f-3356-4e27-a765-944735d6b6d4 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.239206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.239858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.240143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.240422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.241115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.243047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.271004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1279/2517] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:20 2022] POST /identity/v3/projects => generated 355 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:20.275714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.287352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdef46-5904-439d-a04e-af47f12b0ada None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.341081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.341746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.342033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.342313 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.343026 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.345484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.364195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1239/2518] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:20 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:20.368152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.378347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7a375-871e-4be7-95ad-74a9535eda70 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.382888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.383536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.383818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.384346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.385068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.386677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.398287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1280/2519] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:20.402315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.412740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e757855-0f08-4e1f-ad05-6b0c3d2c6481 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.417365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.418233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.418516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.418824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] PATH_INFO: `/v3/projects/bf53d1a9e9ab4edf888ebdbe8a019410/users/eab7fd6af687402c8b3b0ffbb9884980/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.434181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf53d1a9e9ab4edf888ebdbe8a019410, user_id=eab7fd6af687402c8b3b0ffbb9884980, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.436877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.449969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1240/2520] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:20 2022] PUT /identity/v3/projects/bf53d1a9e9ab4edf888ebdbe8a019410/users/eab7fd6af687402c8b3b0ffbb9884980/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:20.454291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.464015 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea4eaf-54e3-462a-9ac3-b2bf5645663d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.519919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c24f7-6252-42fb-bfff-b8438e2b0451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.520584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c24f7-6252-42fb-bfff-b8438e2b0451 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.520869 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c24f7-6252-42fb-bfff-b8438e2b0451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.521149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c24f7-6252-42fb-bfff-b8438e2b0451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.521873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c24f7-6252-42fb-bfff-b8438e2b0451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.523494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c24f7-6252-42fb-bfff-b8438e2b0451 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.533065 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1281/2521] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:20.537228 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.547306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab1e3d-eb94-4e5d-903d-fa532108d459 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.551941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.552806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.553087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.553360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] PATH_INFO: `/v3/projects/bf53d1a9e9ab4edf888ebdbe8a019410/users/eab7fd6af687402c8b3b0ffbb9884980/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.555331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf53d1a9e9ab4edf888ebdbe8a019410, user_id=eab7fd6af687402c8b3b0ffbb9884980, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.557949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.568525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1241/2522] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:20 2022] PUT /identity/v3/projects/bf53d1a9e9ab4edf888ebdbe8a019410/users/eab7fd6af687402c8b3b0ffbb9884980/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:20.580264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.580771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.580886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.606202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] MFA Rules not processed for user `eab7fd6af687402c8b3b0ffbb9884980`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:20.607433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:20.648006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:20.675492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1282/2523] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:20.686492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.688707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:20.748992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30002a1-c9d3-4379-bd15-f55891cb5898 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.753142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.753811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.754094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.754372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.764140 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.765611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.776185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1242/2524] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:20.847221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:20.848627 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:20.860923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a0902-8f7f-487f-a0ae-33db3944804d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:20.864563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:20.865204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:20.865491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:20.865772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:20.927978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:20.929407 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:20.939028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1283/2525] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:21.372253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:21.380860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587d1a3-bc6b-4eb1-8ca6-ac763608a241 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:21.384723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:21.385353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:21.385636 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:21.385905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:21.387693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:21.438968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:21.466235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:21.467727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:21.486530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1243/2526] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:28.492955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:28.495266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:28.568659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c36f71-fb18-4178-9437-863170ceb37d service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:28.573045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:28.573729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:28.574006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:28.574401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:28.576718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:28.658318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:28.659905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:28.671042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1284/2527] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:31.483397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=17 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:32.396746 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:32.398391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:32.414846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59595-137f-423b-84cd-15a92bdfd226 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:32.419311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:32.420174 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:32.420450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:32.420719 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] PATH_INFO: `/v3/users/0884a33a47e448e7826f257880a28964` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:32.421890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0884a33a47e448e7826f257880a28964)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:32.423838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:32.468509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0884a33a47e448e7826f257880a28964'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.475287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0884a33a47e448e7826f257880a28964'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.486699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0884a33a47e448e7826f257880a28964'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.487523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0884a33a47e448e7826f257880a28964'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.488129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0b7435bc-2b3f-49d9-aaff-e89a8b9102a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0884a33a47e448e7826f257880a28964'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.500264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1244/2528] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:32 2022] DELETE /identity/v3/users/0884a33a47e448e7826f257880a28964 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:32.672997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:32.689055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e18a0b-0a65-41a5-8201-2cea7f797723 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:32.721287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:32.753735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:32.755521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:32.755808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:32.756086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] PATH_INFO: `/v3/projects/e43ca8a5c9f2494fbb9c5c5509c5c79b` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:32.757279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43ca8a5c9f2494fbb9c5c5509c5c79b)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:32.759521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:32.803866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43ca8a5c9f2494fbb9c5c5509c5c79b', 'request_id': 'req-f7b35436-2443-403e-a0f4-2effcb64abb7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.810006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43ca8a5c9f2494fbb9c5c5509c5c79b', 'request_id': 'req-f7b35436-2443-403e-a0f4-2effcb64abb7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.816202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43ca8a5c9f2494fbb9c5c5509c5c79b', 'request_id': 'req-f7b35436-2443-403e-a0f4-2effcb64abb7'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.829683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] The token cache is being invalidate because project e43ca8a5c9f2494fbb9c5c5509c5c79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:32.829974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.830600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f7b35436-2443-403e-a0f4-2effcb64abb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:32.832187 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1285/2529] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:32 2022] DELETE /identity/v3/projects/e43ca8a5c9f2494fbb9c5c5509c5c79b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:32.837046 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:32.837339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:32.837642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:32.855218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:32.856422 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:32.916907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1245/2530] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:32.924918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:32.925252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:32.925777 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:32.945164 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:32.946547 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:32.961021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1286/2531] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:32.965025 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:32.975549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7194e6-db0a-4413-8050-bc554432d235 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:32.980343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:32.981010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:32.981298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:32.981579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:32.982454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:32.982800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:32.984396 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.001235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1246/2532] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:33.005858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.016101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19dc24-43a6-4d41-aded-089522fe7f7d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.021481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.022119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.022401 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.022706 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.023410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.025440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.061840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1287/2533] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:33 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:33.062977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.074184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf5094-68fa-4f82-9070-fedb2891d798 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.148226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.148995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.149332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.149666 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.150453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.152489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.162066 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:33.183363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1247/2534] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:33 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:33.187852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.200328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e08438-5197-4727-a020-f5ee9404d873 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.205474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.206152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.206438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.206960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.207708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.209426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.225756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1288/2535] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:33.231452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.242159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc4a94-b583-4bd1-9890-21168b815f24 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.246769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d630d23-452a-4a29-b87a-1e98a261ecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.247417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630d23-452a-4a29-b87a-1e98a261ecf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.247700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630d23-452a-4a29-b87a-1e98a261ecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.247978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d630d23-452a-4a29-b87a-1e98a261ecf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.248696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d630d23-452a-4a29-b87a-1e98a261ecf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.250273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d630d23-452a-4a29-b87a-1e98a261ecf1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.260615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1248/2536] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:33.264685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.275052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83388f37-e733-4504-8112-3e89ff3f49b6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.279746 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.280616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.280905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.281185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] PATH_INFO: `/v3/projects/241ca5af88054d47bf4b944aa4eedb10/users/a24f89ac34704a9e952aa2892a91e5cc/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.296476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241ca5af88054d47bf4b944aa4eedb10, user_id=a24f89ac34704a9e952aa2892a91e5cc, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.299778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.312112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1289/2537] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:33 2022] PUT /identity/v3/projects/241ca5af88054d47bf4b944aa4eedb10/users/a24f89ac34704a9e952aa2892a91e5cc/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:33.323123 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.323433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.323711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.349618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] MFA Rules not processed for user `a24f89ac34704a9e952aa2892a91e5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:33.350867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:33.413534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1249/2538] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:33.417895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.471490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:33.483064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd93bd-65a6-4de9-87ac-70422a8d3ee5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.489329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.490012 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.490356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.490769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.491481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.493598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.520913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1290/2539] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:33 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:33.525195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.536765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5959a-1b3e-47c4-9748-8e8153c4ea36 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.599428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.600249 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.600542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.600823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.601514 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.603471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.622070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1250/2540] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:33 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:33.626024 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.636684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ef2c7-1fce-4d3e-95bc-23027686c23a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.641302 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.641960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.642818 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.643102 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.643827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.645421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.656013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1291/2541] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:33.660604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.670481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175e6c9-83ac-43cc-b220-f8a72581ddfb admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.675124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.675986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.676269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.676546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] PATH_INFO: `/v3/projects/2f0c340a91d0401fb51475e28ad743e6/users/9cb458867d694b4eb0beb5e444af4606/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.692028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0c340a91d0401fb51475e28ad743e6, user_id=9cb458867d694b4eb0beb5e444af4606, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.694903 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.709386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1251/2542] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:33 2022] PUT /identity/v3/projects/2f0c340a91d0401fb51475e28ad743e6/users/9cb458867d694b4eb0beb5e444af4606/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:33.713259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.723803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7a0b-6c67-473a-aff3-66939f510d68 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.779056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895d520-c52e-4ed6-9918-cc50375d6c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.779734 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d520-c52e-4ed6-9918-cc50375d6c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.780020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d520-c52e-4ed6-9918-cc50375d6c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.780297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895d520-c52e-4ed6-9918-cc50375d6c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.781018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895d520-c52e-4ed6-9918-cc50375d6c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.782649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895d520-c52e-4ed6-9918-cc50375d6c05 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.792720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1292/2543] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:33.797024 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.807263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004a1f4-1e24-49b3-89ee-9e939116f92b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:33.811799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:33.812670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.812948 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.813224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] PATH_INFO: `/v3/projects/2f0c340a91d0401fb51475e28ad743e6/users/9cb458867d694b4eb0beb5e444af4606/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.815272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0c340a91d0401fb51475e28ad743e6, user_id=9cb458867d694b4eb0beb5e444af4606, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:33.817884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:33.828000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1252/2544] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:33 2022] PUT /identity/v3/projects/2f0c340a91d0401fb51475e28ad743e6/users/9cb458867d694b4eb0beb5e444af4606/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:33.838282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9548e-6437-4257-a233-9da381309422 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:33.838572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9548e-6437-4257-a233-9da381309422 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:33.838873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9548e-6437-4257-a233-9da381309422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:33.864812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cbf9548e-6437-4257-a233-9da381309422 None None] MFA Rules not processed for user `9cb458867d694b4eb0beb5e444af4606`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:33.865993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cbf9548e-6437-4257-a233-9da381309422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:33.932792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1293/2545] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:33.944214 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:33.945910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:33.968546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:34.003564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e397-a1ef-432d-bec9-a575564f6497 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:34.007954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:34.008600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:34.008885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:34.009428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:34.019053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:34.020477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:34.029809 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1253/2546] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:33 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:34.343581 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9548e-6437-4257-a233-9da381309422 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:34.347058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cbf9548e-6437-4257-a233-9da381309422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:34.415811 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:34.421899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9548e-6437-4257-a233-9da381309422 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:34.426497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:34.427387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:34.427771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:34.428143 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:34.430255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:34.455491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:34.507278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:34.509346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2a5fd-bad3-43cc-8a7e-7bcb914a25d1 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:34.522859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec6291-98f3-41ad-91ee-b4b9c2e449d2 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:34.527276 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1294/2547] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:34.527490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:34.528593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:34.528884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:34.529159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] PATH_INFO: `/v3/users/a24f89ac34704a9e952aa2892a91e5cc` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:34.530368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24f89ac34704a9e952aa2892a91e5cc)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:34.532423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:34.594241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24f89ac34704a9e952aa2892a91e5cc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:34.604681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24f89ac34704a9e952aa2892a91e5cc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:34.619489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24f89ac34704a9e952aa2892a91e5cc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:34.620190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24f89ac34704a9e952aa2892a91e5cc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:34.620812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24f89ac34704a9e952aa2892a91e5cc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:34.641754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1254/2548] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:34 2022] DELETE /identity/v3/users/a24f89ac34704a9e952aa2892a91e5cc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:34.645370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:34.645666 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:34.645950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:34.663829 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:34.665078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:34.734701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1295/2549] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:34.744723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:34.746157 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:34.792865 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:34.807290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d7acb-068a-4758-a9f1-9720c3775ab1 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:34.810955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:34.811601 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:34.811884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:34.812161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:34.821241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:34.822693 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:34.832474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1255/2550] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:35.120175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:35.121974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:35.136159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39363292-e239-4d1d-bfaf-6c80e9befa19 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:35.143457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:35.144554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:35.145036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:35.145486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] PATH_INFO: `/v3/projects/241ca5af88054d47bf4b944aa4eedb10` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:35.146953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241ca5af88054d47bf4b944aa4eedb10)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:35.149372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:35.203842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241ca5af88054d47bf4b944aa4eedb10', 'request_id': 'req-7177fb96-e602-4af2-9a19-81c35da027b6'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.210793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241ca5af88054d47bf4b944aa4eedb10', 'request_id': 'req-7177fb96-e602-4af2-9a19-81c35da027b6'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.218080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241ca5af88054d47bf4b944aa4eedb10', 'request_id': 'req-7177fb96-e602-4af2-9a19-81c35da027b6'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.232660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] The token cache is being invalidate because project 241ca5af88054d47bf4b944aa4eedb10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:35.232961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.233593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.235504 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1296/2551] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:35 2022] DELETE /identity/v3/projects/241ca5af88054d47bf4b944aa4eedb10 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:35.239483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:35.249184 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a4794-a9c3-4c28-8342-7acaa246469a service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:35.309790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:35.310709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:35.310982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:35.311257 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] PATH_INFO: `/v3/users/9cb458867d694b4eb0beb5e444af4606` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:35.312439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cb458867d694b4eb0beb5e444af4606)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:35.314576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:35.365124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cb458867d694b4eb0beb5e444af4606'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.375816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cb458867d694b4eb0beb5e444af4606'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.388755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cb458867d694b4eb0beb5e444af4606'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.389401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cb458867d694b4eb0beb5e444af4606'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.390008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b259bf5c-dfac-4297-b4b4-519e0e81af81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cb458867d694b4eb0beb5e444af4606'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.403354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1256/2552] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:35 2022] DELETE /identity/v3/users/9cb458867d694b4eb0beb5e444af4606 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:35.693046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:35.694895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:35.710752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177fb96-e602-4af2-9a19-81c35da027b6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:35.729656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:35.777424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:35.778383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:35.778766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:35.779114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] PATH_INFO: `/v3/projects/2f0c340a91d0401fb51475e28ad743e6` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:35.781030 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0c340a91d0401fb51475e28ad743e6)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:35.785683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:35.848009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0c340a91d0401fb51475e28ad743e6', 'request_id': 'req-e9582f82-c064-44de-91fa-9dd6bda7620b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.856983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0c340a91d0401fb51475e28ad743e6', 'request_id': 'req-e9582f82-c064-44de-91fa-9dd6bda7620b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.864039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0c340a91d0401fb51475e28ad743e6', 'request_id': 'req-e9582f82-c064-44de-91fa-9dd6bda7620b'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.884957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] The token cache is being invalidate because project 2f0c340a91d0401fb51475e28ad743e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:35.885323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.886055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-e9582f82-c064-44de-91fa-9dd6bda7620b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:35.888055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1297/2553] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:35 2022] DELETE /identity/v3/projects/2f0c340a91d0401fb51475e28ad743e6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:35.894081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:35.894594 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:35.894808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:35.918090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:35.919489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:35.989286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1257/2554] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:35.996951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:35.997311 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:35.997665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.019440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:36.020775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.035636 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1298/2555] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:36.036606 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.045575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:36.051434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51faa5b7-2f1b-4768-9a32-52f511799ec1 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.059571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.060844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.062120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.063947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.067089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.067454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.068347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:36.068740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.070530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.079821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82464110-59ef-42d2-8a55-c87d87048f65 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.085253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.086507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.087084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.087554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] PATH_INFO: `/v3/users/5384f8e641fd4837a0d1d56584f8559a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.089096 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5384f8e641fd4837a0d1d56584f8559a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.090661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1258/2556] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:15:36.092109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.094884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.106744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88372b-8362-4c0c-ba5f-9ca392633ab2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.112622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.113484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.113928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.114439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.115348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.117627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.158234 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1259/2557] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:36 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:36.159375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.179237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5384f8e641fd4837a0d1d56584f8559a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.196194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5384f8e641fd4837a0d1d56584f8559a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.214211 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5384f8e641fd4837a0d1d56584f8559a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.215897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5384f8e641fd4837a0d1d56584f8559a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.216563 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7c6cc1b8-e1d5-4ce8-bcc2-8c07ff3fa2d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5384f8e641fd4837a0d1d56584f8559a'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.237984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1299/2558] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:36 2022] DELETE /identity/v3/users/5384f8e641fd4837a0d1d56584f8559a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:36.242987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.244168 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.244892 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.264217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c471d6a-68aa-4021-8f34-2db834cb4814 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.267858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:36.269505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.286562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:36.357210 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1300/2559] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:36.360493 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.361743 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.362032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.362311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.363053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.365646 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.366695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.367600 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.394600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1260/2560] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:36 2022] POST /identity/v3/users => generated 296 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:36.395397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.398145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.413989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b61e03-7dd0-4c92-bb87-3daee5cb76d2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.427435 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.428316 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.428791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.429241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.430193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.432712 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.448781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1261/2561] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:36.452633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd0e1d-3f0f-4279-8bce-1a276eb1b6dc None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.452797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.456705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.457363 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.457642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.457919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.468109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.470145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.478035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f51c2-b9b8-4439-9a30-24e03cd9cb9f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.483233 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1301/2562] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:15:36.485287 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c414d-7298-45f0-ae51-9f8d46fccd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.486649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c414d-7298-45f0-ae51-9f8d46fccd07 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.487013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c414d-7298-45f0-ae51-9f8d46fccd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.487360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c414d-7298-45f0-ae51-9f8d46fccd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.488354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c414d-7298-45f0-ae51-9f8d46fccd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.490078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c414d-7298-45f0-ae51-9f8d46fccd07 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.506895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1262/2563] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:36.507374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.519345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f00d7-14ba-48e1-8d7e-921acd1bd465 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.525202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.526273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.526661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.526972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] PATH_INFO: `/v3/projects/90e45f36590e48afa96451fd7ab2733e/users/5b2e567076c841ebb2c865e17ae25f6f/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.548141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e45f36590e48afa96451fd7ab2733e, user_id=5b2e567076c841ebb2c865e17ae25f6f, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.552299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.571942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1302/2564] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:36 2022] PUT /identity/v3/projects/90e45f36590e48afa96451fd7ab2733e/users/5b2e567076c841ebb2c865e17ae25f6f/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:36.585061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.585447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.585976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.618649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] MFA Rules not processed for user `5b2e567076c841ebb2c865e17ae25f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:36.620979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.662426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:36.716665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1263/2565] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:36.721436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.802727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296003ba-1264-4f45-ac1f-013ad532c3f0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.816411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.824159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.824878 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.825172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.826921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.830584 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.840662 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.842754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.866765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4c321-cf2e-4569-9b7d-6feb46cd6334 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:36.877938 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:36.880048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:36.880525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:36.880904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] PATH_INFO: `/v3/projects/e4b567c3dd80469e8d7302f2f551aef3` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:36.882478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b567c3dd80469e8d7302f2f551aef3)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:36.884757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:36.894339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1303/2566] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:36 2022] POST /identity/v3/projects => generated 393 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:36.895342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.931880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:36.945408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b567c3dd80469e8d7302f2f551aef3', 'request_id': 'req-5713bfed-394a-4793-9f89-709e2771cf12'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.954689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b567c3dd80469e8d7302f2f551aef3', 'request_id': 'req-5713bfed-394a-4793-9f89-709e2771cf12'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.969844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b567c3dd80469e8d7302f2f551aef3', 'request_id': 'req-5713bfed-394a-4793-9f89-709e2771cf12'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.985637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] The token cache is being invalidate because project e4b567c3dd80469e8d7302f2f551aef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:36.985963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.986639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:36.988501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1264/2567] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:36 2022] DELETE /identity/v3/projects/e4b567c3dd80469e8d7302f2f551aef3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:36.992613 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:36.994154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:36.998958 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054f487-8d48-4ca4-bdb4-838fb14e1d6a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.059131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.059816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.060103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.060379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.061076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.063028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.077277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713bfed-394a-4793-9f89-709e2771cf12 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.083148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.084035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.084324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.084603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] PATH_INFO: `/v3/users/eab7fd6af687402c8b3b0ffbb9884980` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.085850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1304/2568] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:36 2022] POST /identity/v3/users => generated 303 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:37.088620 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab7fd6af687402c8b3b0ffbb9884980)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.089525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:37.090826 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.091248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.105194 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786b4b-ef51-40b3-b900-f60d63a2e179 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.109929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.110602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.110966 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.111242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.111973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.113569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.124549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1305/2569] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:37.129562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:37.143921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01a3f7-e1e5-430e-b03a-62459deee5fa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.149243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.150141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.150432 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.150741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] PATH_INFO: `/v3/projects/8acfcaf7e5cf4e0daddd4c32bff45616/users/88d97c4001d84745965b48f302a1d4ff/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.161308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab7fd6af687402c8b3b0ffbb9884980'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.167489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8acfcaf7e5cf4e0daddd4c32bff45616, user_id=88d97c4001d84745965b48f302a1d4ff, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.170916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.171527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab7fd6af687402c8b3b0ffbb9884980'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.177888 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:37.191041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab7fd6af687402c8b3b0ffbb9884980'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.192145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab7fd6af687402c8b3b0ffbb9884980'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.192270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1306/2570] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:37 2022] PUT /identity/v3/projects/8acfcaf7e5cf4e0daddd4c32bff45616/users/88d97c4001d84745965b48f302a1d4ff/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:37.192663 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab7fd6af687402c8b3b0ffbb9884980'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.196565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:37.198385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.210723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1265/2571] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:36 2022] DELETE /identity/v3/users/eab7fd6af687402c8b3b0ffbb9884980 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:37.286831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257a910-0471-487d-a9d5-4012747f68b9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.327466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:37.382000 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e12e85-0f2b-40dc-81ae-44333df4525e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.383502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e12e85-0f2b-40dc-81ae-44333df4525e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.384425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e12e85-0f2b-40dc-81ae-44333df4525e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.384719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e12e85-0f2b-40dc-81ae-44333df4525e admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.385477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e12e85-0f2b-40dc-81ae-44333df4525e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.387435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e12e85-0f2b-40dc-81ae-44333df4525e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.403317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1307/2572] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:37.403890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:37.420253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f577113-8be6-4239-8819-c6d702db309b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.426282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.427299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.427656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.428027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] PATH_INFO: `/v3/projects/8acfcaf7e5cf4e0daddd4c32bff45616/users/88d97c4001d84745965b48f302a1d4ff/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.430290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8acfcaf7e5cf4e0daddd4c32bff45616, user_id=88d97c4001d84745965b48f302a1d4ff, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.433001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.459094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1266/2573] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:37 2022] PUT /identity/v3/projects/8acfcaf7e5cf4e0daddd4c32bff45616/users/88d97c4001d84745965b48f302a1d4ff/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:37.469873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.470216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.470699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.523925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] MFA Rules not processed for user `88d97c4001d84745965b48f302a1d4ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:37.527766 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.540372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:37.542987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.648492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3047a-c3af-4e5a-8c24-8a6b7998d7f5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.653781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.654969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.656151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.656956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] PATH_INFO: `/v3/projects/bf53d1a9e9ab4edf888ebdbe8a019410` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.662121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf53d1a9e9ab4edf888ebdbe8a019410)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:37.670882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:37.730659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1308/2574] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:37.744186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:37.747280 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.756537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf53d1a9e9ab4edf888ebdbe8a019410', 'request_id': 'req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.765740 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf53d1a9e9ab4edf888ebdbe8a019410', 'request_id': 'req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.776282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf53d1a9e9ab4edf888ebdbe8a019410', 'request_id': 'req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.792796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] The token cache is being invalidate because project bf53d1a9e9ab4edf888ebdbe8a019410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:37.793106 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.796913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-0f0ba52f-2d38-4e09-ac33-5ce0f9a01d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:37.802548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1267/2575] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:37 2022] DELETE /identity/v3/projects/bf53d1a9e9ab4edf888ebdbe8a019410 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:37.808895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4378-4aaa-40f1-a032-12da06fe996c None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.809058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4378-4aaa-40f1-a032-12da06fe996c None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.809340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4378-4aaa-40f1-a032-12da06fe996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.831466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2f3b4378-4aaa-40f1-a032-12da06fe996c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:37.832697 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2f3b4378-4aaa-40f1-a032-12da06fe996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.853987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab10278-8484-43df-bbc2-b1e11710bf21 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:37.913800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:37.935142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:37.935376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1268/2576] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:37.935833 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.936118 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.936394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.938491 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:37.943629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:37.943981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:37.944269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:37.968576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:37.987121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:37.989426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.006718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1269/2577] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:38.007829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.020912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adffd14-3842-4176-9f42-91e4cba04403 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.028253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.029672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.029957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.030235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.030444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.031008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:38.031318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.032121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.033550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.045327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1309/2578] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:38.055364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1270/2579] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:38.059908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.074142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01601-0c06-4bef-b0fe-a62ed6b7b8b9 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.080372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.081327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.081614 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.081891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.083041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.087291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.106405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.108456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.122971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5b1d5-e956-41eb-b805-63c519419dff admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.139931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.141434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.141723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.142016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.146672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1310/2580] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/projects => generated 389 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:38.148293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.148587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.242911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.244443 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.252236 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d089-921a-4144-9c37-1dcbafad2fa8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.256319 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1271/2581] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:38.257242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.257881 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.258182 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.258457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.258897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.260453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.268883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.270960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.282037 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52850-9658-40ed-862a-97abcc209795 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.289599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.290885 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.291170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.291446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] PATH_INFO: `/v3/users/d9cab6f67cd34c46886d036ba170331e` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.293226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1311/2582] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:38.297697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.308020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f4b2b-bc47-48ef-b62b-e1e8d8cc3cd5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.309923 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9cab6f67cd34c46886d036ba170331e)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.312192 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.313315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.314936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.315408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.316005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.317097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.319727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.333165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1312/2583] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:38.338792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.351149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df615f-3532-42a1-9a42-76f18f89ad73 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.358020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.359049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.359501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.359763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.360497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.362147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.373120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1313/2584] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:38.376642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9cab6f67cd34c46886d036ba170331e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.377161 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.386586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9cab6f67cd34c46886d036ba170331e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.391900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6f252-7b68-41df-a256-57fcf4ebd81f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.400492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.401644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.402077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.402543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] PATH_INFO: `/v3/projects/613f42da22ea4ec0b7b849045a786e20/users/dc77edc53c774b05a5a3fb9ff17f0069/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.409928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9cab6f67cd34c46886d036ba170331e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.411132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9cab6f67cd34c46886d036ba170331e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.411943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9212e166-dfca-46cb-af53-7e30a58f88b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9cab6f67cd34c46886d036ba170331e'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.427832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613f42da22ea4ec0b7b849045a786e20, user_id=dc77edc53c774b05a5a3fb9ff17f0069, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.433006 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.436689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1272/2585] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:38 2022] DELETE /identity/v3/users/d9cab6f67cd34c46886d036ba170331e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:38.458711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1314/2586] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:38 2022] PUT /identity/v3/projects/613f42da22ea4ec0b7b849045a786e20/users/dc77edc53c774b05a5a3fb9ff17f0069/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:38.466150 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.466441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.466763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.508451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] MFA Rules not processed for user `dc77edc53c774b05a5a3fb9ff17f0069`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:38.509920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.533622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:38.614269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1273/2587] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:38.615389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.617249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.654473 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.659562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.674128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:38.702320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b1ebc-a5b5-4e36-807c-16b2624cb587 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.709239 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.709934 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.710217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.710502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.711271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.713214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.740308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1315/2588] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:38.744354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d44478-d1c0-4a03-940f-d39838c867e5 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.744732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.801895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.802857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.803136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.803412 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] PATH_INFO: `/v3/projects/0a2703a8b75043e0ab0656482fa9ddeb` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.807283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5875c-16a3-49cd-a457-972e42b78ca3 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.811897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.812562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.812842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.813116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.813797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.814915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2703a8b75043e0ab0656482fa9ddeb)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.816324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.816911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.839367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1316/2589] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:38.840391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.851219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f55017-2638-4ce6-abad-231484db395d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.856130 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.856814 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.857220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.857506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.858235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.859851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.868786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2703a8b75043e0ab0656482fa9ddeb', 'request_id': 'req-765eed2a-907e-40d9-8b7e-149697f2c073'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.873633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1317/2590] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:38.874711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.879238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2703a8b75043e0ab0656482fa9ddeb', 'request_id': 'req-765eed2a-907e-40d9-8b7e-149697f2c073'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.885125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2703a8b75043e0ab0656482fa9ddeb', 'request_id': 'req-765eed2a-907e-40d9-8b7e-149697f2c073'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.885262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d87a2-d75b-49b0-8887-58b4b02044e8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:38.886838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:38.899371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] The token cache is being invalidate because project 0a2703a8b75043e0ab0656482fa9ddeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:38.899786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.900453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-765eed2a-907e-40d9-8b7e-149697f2c073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:38.902064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1274/2591] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:38 2022] DELETE /identity/v3/projects/0a2703a8b75043e0ab0656482fa9ddeb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:38.906723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc908-ba35-4431-aede-a7985c6162d4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.907202 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc908-ba35-4431-aede-a7985c6162d4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.907515 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc908-ba35-4431-aede-a7985c6162d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.925283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2c2fc908-ba35-4431-aede-a7985c6162d4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:38.926475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2c2fc908-ba35-4431-aede-a7985c6162d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.945756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:38.946575 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.946885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.947167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] PATH_INFO: `/v3/projects/aaf61bb500ff4e488145f6b45be7683a/users/2e5251a290a643bb8965e3424e5674c1/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:38.961443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf61bb500ff4e488145f6b45be7683a, user_id=2e5251a290a643bb8965e3424e5674c1, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:38.964153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:38.977828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1318/2592] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:38 2022] PUT /identity/v3/projects/aaf61bb500ff4e488145f6b45be7683a/users/2e5251a290a643bb8965e3424e5674c1/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:38.981679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:38.983323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:38.986470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1275/2593] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:38.994082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:38.994675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:38.994675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.013523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:39.014783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.019040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:39.054873 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981577-31e0-4674-a750-7537a9a4f084 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.059678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.060352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.060640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.060925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.061652 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.063294 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.073845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1319/2594] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:39.075958 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1276/2595] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:39.077680 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.080093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.087752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bb11d-76ac-4938-aed4-bfab989fa81a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.088731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4f87f-a775-4e63-8d1d-c496e3a8bd2d None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.092590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db44833-bbe1-40f9-a8cc-7c0b77f9b120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.093433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.093554 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44833-bbe1-40f9-a8cc-7c0b77f9b120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.093722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44833-bbe1-40f9-a8cc-7c0b77f9b120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.094005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db44833-bbe1-40f9-a8cc-7c0b77f9b120 admin admin] PATH_INFO: `/v3/projects/aaf61bb500ff4e488145f6b45be7683a/users/2e5251a290a643bb8965e3424e5674c1/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.094131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.094348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.094642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.095391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:39.095699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.096039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db44833-bbe1-40f9-a8cc-7c0b77f9b120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf61bb500ff4e488145f6b45be7683a, user_id=2e5251a290a643bb8965e3424e5674c1, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.097325 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.098678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db44833-bbe1-40f9-a8cc-7c0b77f9b120 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.108867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1277/2596] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:15:39.112909 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1320/2597] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:39 2022] PUT /identity/v3/projects/aaf61bb500ff4e488145f6b45be7683a/users/2e5251a290a643bb8965e3424e5674c1/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:39.113467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.122571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.122903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.123203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.123363 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ddd9b-1e52-4e8c-ba35-1e289ab46dcc admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.152035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] MFA Rules not processed for user `2e5251a290a643bb8965e3424e5674c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:39.153842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.189524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.190191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.190483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.190801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.191523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.193602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.224456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1278/2598] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:39.225376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.226094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1321/2599] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:39.230883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:39.238348 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.239940 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.292587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d8002-2eae-4ad4-ba44-c18cbaf40a36 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.297799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.299160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.299480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.299745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.300436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.302354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.306328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7dabf-2244-403b-8f82-4eb6de88b788 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.322027 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1279/2600] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:39.326050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.335919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd09f7c-eeab-406a-ac46-6a5df7cb67b0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.340841 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.341571 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.341930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.342282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.343166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.344988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.358220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6224b-91fe-45a8-a2e9-8db56cf970d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.358426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1280/2601] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:39.358541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6224b-91fe-45a8-a2e9-8db56cf970d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.358541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6224b-91fe-45a8-a2e9-8db56cf970d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.358541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6224b-91fe-45a8-a2e9-8db56cf970d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.359216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.361738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:39.369452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36639590-ea41-47b9-a62e-2b9a0bb6f874 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.374347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.375013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.375301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.375582 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.376302 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.377908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.392068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1281/2602] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:39.392990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.402343 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3e89-2583-454a-9469-4c7d8acab4df admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.406997 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.408477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.408656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.408937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] PATH_INFO: `/v3/projects/106cf00ba7ed4643ba4aeb927ba8787a/users/4ce82d4e21b84460b5896202da5ec8dd/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.422394 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6224b-91fe-45a8-a2e9-8db56cf970d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.423219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] RBAC: Authorizing `identity:create_grant(project_id=106cf00ba7ed4643ba4aeb927ba8787a, user_id=4ce82d4e21b84460b5896202da5ec8dd, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.425963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6224b-91fe-45a8-a2e9-8db56cf970d5 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.427270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.435031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1322/2603] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:15:39.439464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1282/2604] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:39 2022] PUT /identity/v3/projects/106cf00ba7ed4643ba4aeb927ba8787a/users/4ce82d4e21b84460b5896202da5ec8dd/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:39.450299 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.450862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.451089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.478713 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] MFA Rules not processed for user `4ce82d4e21b84460b5896202da5ec8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:39.480029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.556534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1323/2605] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:39.561044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.592944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.640261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108496b1-0d2d-4bfb-a2aa-27e4e2f3533a admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.646467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.647311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.649104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.649104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.649104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.650609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.672023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365322a-6e26-4f11-a6e2-27ed93e2de73 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.678022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.679119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.679403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.679675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] PATH_INFO: `/v3/users/dc77edc53c774b05a5a3fb9ff17f0069` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.680888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc77edc53c774b05a5a3fb9ff17f0069)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.682823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.684440 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1283/2606] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:39.688413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.727605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:39.730941 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:39.745960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc77edc53c774b05a5a3fb9ff17f0069'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:39.753821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc77edc53c774b05a5a3fb9ff17f0069'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:39.760943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1eba4-4b04-4d1d-a7ff-255a1ad6be0c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.766156 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.766858 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.767661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.767941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.768747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.769990 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc77edc53c774b05a5a3fb9ff17f0069'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:39.770777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.771014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc77edc53c774b05a5a3fb9ff17f0069'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:39.771803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-7463a592-1daa-464d-a966-6c90c8a1e052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc77edc53c774b05a5a3fb9ff17f0069'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:39.786905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1324/2607] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:39 2022] DELETE /identity/v3/users/dc77edc53c774b05a5a3fb9ff17f0069 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:39.791070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.791502 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.791885 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.793546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1284/2608] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/users => generated 308 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:39.797081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.799001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.812456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75f62f-57d9-4430-9a91-b77559893596 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.816574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:39.817792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.872248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.873072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.873485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.873875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.874635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.876251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.879524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1325/2609] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:39.889562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.890320 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1285/2610] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:39.891210 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.891512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:39.901654 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac1611-3c8b-40cb-892f-a73bef4aea6f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:39.908709 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:39.910229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:39.910596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:39.911005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] PATH_INFO: `/v3/projects/0342793f5ac643a0802d85ceaaee8322/users/bd4a02de5b924b8d93dcdca7c7381949/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:39.928550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0342793f5ac643a0802d85ceaaee8322, user_id=bd4a02de5b924b8d93dcdca7c7381949, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:39.931324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:39.945258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1286/2611] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:39 2022] PUT /identity/v3/projects/0342793f5ac643a0802d85ceaaee8322/users/bd4a02de5b924b8d93dcdca7c7381949/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:39.949093 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:39.953729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da3d06-0da8-4283-8a4d-62c518f8e604 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.010438 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.011238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.011533 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.011810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.013805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a3fa5-0505-448c-a007-dc00d426a854 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.019522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173264e-00d3-4c27-b5f1-447c65ff401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.020195 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173264e-00d3-4c27-b5f1-447c65ff401b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.020486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173264e-00d3-4c27-b5f1-447c65ff401b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.020767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173264e-00d3-4c27-b5f1-447c65ff401b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.021537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173264e-00d3-4c27-b5f1-447c65ff401b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.022369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.023180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173264e-00d3-4c27-b5f1-447c65ff401b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.023901 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.034859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1326/2612] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:40.035137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1287/2613] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:40.039987 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:40.051076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53626d-6bd0-4145-acdd-2a87fc8e9a47 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.056112 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.057034 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.057316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.057595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] PATH_INFO: `/v3/projects/0342793f5ac643a0802d85ceaaee8322/users/bd4a02de5b924b8d93dcdca7c7381949/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.059659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0342793f5ac643a0802d85ceaaee8322, user_id=bd4a02de5b924b8d93dcdca7c7381949, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.062602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.066454 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:40.082684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1327/2614] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:40 2022] PUT /identity/v3/projects/0342793f5ac643a0802d85ceaaee8322/users/bd4a02de5b924b8d93dcdca7c7381949/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:40.087511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.087894 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.088067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.119910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] MFA Rules not processed for user `bd4a02de5b924b8d93dcdca7c7381949`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:40.121144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:40.146786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:40.219073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1288/2615] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:40.226221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:40.228493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:40.303010 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893c6a7-5a01-467f-b9e6-db02496919aa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.307110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.307882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.308184 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.308557 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.319701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.321566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.332840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1328/2616] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:40.349727 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:40.351353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:40.437203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41921cc0-a7f7-48ac-906b-aa2ec06254ce None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.442256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.443152 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.443433 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.443783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] PATH_INFO: `/v3/projects/613f42da22ea4ec0b7b849045a786e20` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.444998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=613f42da22ea4ec0b7b849045a786e20)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.448331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.495396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:40.508219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b478c22-8dfd-45d5-bfbf-eecf2adf3135 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.511792 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613f42da22ea4ec0b7b849045a786e20', 'request_id': 'req-edcd58c0-ec86-492f-a2b1-cc553ff51c37'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.519060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613f42da22ea4ec0b7b849045a786e20', 'request_id': 'req-edcd58c0-ec86-492f-a2b1-cc553ff51c37'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.525491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613f42da22ea4ec0b7b849045a786e20', 'request_id': 'req-edcd58c0-ec86-492f-a2b1-cc553ff51c37'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.541293 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] The token cache is being invalidate because project 613f42da22ea4ec0b7b849045a786e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:40.541785 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.542430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.544366 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1289/2617] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:40 2022] DELETE /identity/v3/projects/613f42da22ea4ec0b7b849045a786e20 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:40.548056 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:40.550947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:40.573487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.574373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.574676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.575495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] PATH_INFO: `/v3/users/4ce82d4e21b84460b5896202da5ec8dd` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.577148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce82d4e21b84460b5896202da5ec8dd)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.580316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.622929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:40.636379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce82d4e21b84460b5896202da5ec8dd'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.642021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd58c0-ec86-492f-a2b1-cc553ff51c37 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.643718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce82d4e21b84460b5896202da5ec8dd'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.647294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.648194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.648463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.648744 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] PATH_INFO: `/v3/users/2e5251a290a643bb8965e3424e5674c1` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.650001 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5251a290a643bb8965e3424e5674c1)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.651978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.656663 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce82d4e21b84460b5896202da5ec8dd'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.657420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce82d4e21b84460b5896202da5ec8dd'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.662944 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-bf47231d-5f4f-4b28-b99a-c7feddbac194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce82d4e21b84460b5896202da5ec8dd'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.675842 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1329/2618] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:40 2022] DELETE /identity/v3/users/4ce82d4e21b84460b5896202da5ec8dd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:40.680339 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.680637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.680926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.699179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:40.700385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:40.702018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5251a290a643bb8965e3424e5674c1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.711124 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5251a290a643bb8965e3424e5674c1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.712567 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:40.724971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5251a290a643bb8965e3424e5674c1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.725687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5251a290a643bb8965e3424e5674c1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.726426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5251a290a643bb8965e3424e5674c1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:40.739842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1290/2619] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:40 2022] DELETE /identity/v3/users/2e5251a290a643bb8965e3424e5674c1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:40.765396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1330/2620] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:40.775276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:40.776756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:40.846451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a2ede-6f57-43a2-96cc-f2ed98fbb366 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:40.850626 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef177ddd-d0b0-41df-a479-d8a36bbbf261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:40.851911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef177ddd-d0b0-41df-a479-d8a36bbbf261 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:40.852203 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef177ddd-d0b0-41df-a479-d8a36bbbf261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:40.852490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef177ddd-d0b0-41df-a479-d8a36bbbf261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:40.917933 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef177ddd-d0b0-41df-a479-d8a36bbbf261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:40.919413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef177ddd-d0b0-41df-a479-d8a36bbbf261 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:40.929629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1291/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:41.115880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.117345 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.129501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f1156-1fa7-4b8e-be25-497d53fb4bce None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.134371 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.135324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.135867 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.136176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] PATH_INFO: `/v3/projects/aaf61bb500ff4e488145f6b45be7683a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.137673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf61bb500ff4e488145f6b45be7683a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.140408 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.202633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf61bb500ff4e488145f6b45be7683a', 'request_id': 'req-a52a0275-4d25-451c-a6f4-5898a7f420fa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.209846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf61bb500ff4e488145f6b45be7683a', 'request_id': 'req-a52a0275-4d25-451c-a6f4-5898a7f420fa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.215761 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf61bb500ff4e488145f6b45be7683a', 'request_id': 'req-a52a0275-4d25-451c-a6f4-5898a7f420fa'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.229489 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] The token cache is being invalidate because project aaf61bb500ff4e488145f6b45be7683a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:41.229781 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.230418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.231999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1331/2622] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:41 2022] DELETE /identity/v3/projects/aaf61bb500ff4e488145f6b45be7683a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:41.236895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d23c6c-3881-4ad3-bc76-c0fa22635893 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.237375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d23c6c-3881-4ad3-bc76-c0fa22635893 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.237478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d23c6c-3881-4ad3-bc76-c0fa22635893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.255497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d1d23c6c-3881-4ad3-bc76-c0fa22635893 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:41.256853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d1d23c6c-3881-4ad3-bc76-c0fa22635893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.279051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:41.301288 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.302768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.332089 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1292/2623] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:41.339803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.340097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.340378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.358853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:41.360071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.368528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1293/2624] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:41.372557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.374253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a0275-4d25-451c-a6f4-5898a7f420fa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.379094 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.380023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.380375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.380651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] PATH_INFO: `/v3/projects/106cf00ba7ed4643ba4aeb927ba8787a` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.381676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8198e0-27a3-44c8-8917-a832ef1eaecd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.381845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=106cf00ba7ed4643ba4aeb927ba8787a)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.383772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.386767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.387409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.387699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.388038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.388725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:41.389148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.390762 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.403142 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1294/2625] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:15:41.407231 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.418180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb20879-ebf9-4f8e-bd41-ec2fdd5fe7fb admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.429245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:41.433220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106cf00ba7ed4643ba4aeb927ba8787a', 'request_id': 'req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.442994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106cf00ba7ed4643ba4aeb927ba8787a', 'request_id': 'req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.449241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106cf00ba7ed4643ba4aeb927ba8787a', 'request_id': 'req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.464361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] The token cache is being invalidate because project 106cf00ba7ed4643ba4aeb927ba8787a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:41.464816 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.465475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.467079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1332/2626] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:41 2022] DELETE /identity/v3/projects/106cf00ba7ed4643ba4aeb927ba8787a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:41.470947 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.472252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.478169 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.478859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.479148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.479423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.480114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.481998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.511252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1295/2627] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:41 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:41.512341 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.514223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.540880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe62ae-9cfa-4ef7-b2a7-34e1f79d79e6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.580791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551a2a7-b9f8-4965-a2ae-0bf4b2016e33 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.586327 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.587032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.587323 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.587600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.588289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.590209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.599049 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.599917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.600197 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.600471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] PATH_INFO: `/v3/users/bd4a02de5b924b8d93dcdca7c7381949` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.602215 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4a02de5b924b8d93dcdca7c7381949)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.604499 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.610451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1296/2628] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:41 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:41.614204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.624692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031d9af-061a-4fe3-9c72-61480824234e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.630010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.630692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.630976 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.631324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.631961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.633672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.639632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:41.646897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1297/2629] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:41.650828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.662454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c26b99-f606-4651-97e0-5cbc82bd1e72 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.667829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.668587 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.668883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.669279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.670045 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.671672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.672041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4a02de5b924b8d93dcdca7c7381949'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.688298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1298/2630] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:41.692972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.694513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4a02de5b924b8d93dcdca7c7381949'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.703012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923951a0-354d-4797-ab30-efc268641d1f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.709623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.710379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4a02de5b924b8d93dcdca7c7381949'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.710517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.710972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.711272 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] PATH_INFO: `/v3/projects/27f02394763340b7be6dcf6152b7d5a3/users/a4509851374348e7884055ed11e1a705/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.711416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4a02de5b924b8d93dcdca7c7381949'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.711997 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-95cb9509-afae-4f72-b59f-148338e4e4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4a02de5b924b8d93dcdca7c7381949'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.721832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:41.728694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1333/2631] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:41 2022] DELETE /identity/v3/users/bd4a02de5b924b8d93dcdca7c7381949 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:41.730878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f02394763340b7be6dcf6152b7d5a3, user_id=a4509851374348e7884055ed11e1a705, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.734384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.750667 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1299/2632] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:41 2022] PUT /identity/v3/projects/27f02394763340b7be6dcf6152b7d5a3/users/a4509851374348e7884055ed11e1a705/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:41.762243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.762737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.763004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.793388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] MFA Rules not processed for user `a4509851374348e7884055ed11e1a705`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:41.794410 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.813247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.815692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.871193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1334/2633] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:41.882910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:41.884618 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:41.894941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42849b36-3286-4854-8857-da59b829dba5 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.900833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.902185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.902572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.903431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] PATH_INFO: `/v3/users/5b2e567076c841ebb2c865e17ae25f6f` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.904534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2e567076c841ebb2c865e17ae25f6f)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.906565 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.952562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00583bfa-ffb3-4f67-b84f-e6c4b63e6cc2 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:41.956685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a91ad-610a-4ca5-b2f6-fae80e88bfc5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6f4dc9555784337b2fd930caea78fde', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:41.957343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a91ad-610a-4ca5-b2f6-fae80e88bfc5 service glance] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:41.957629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a91ad-610a-4ca5-b2f6-fae80e88bfc5 service glance] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:41.957908 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a91ad-610a-4ca5-b2f6-fae80e88bfc5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:41.958269 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2e567076c841ebb2c865e17ae25f6f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.967032 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2e567076c841ebb2c865e17ae25f6f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.967316 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a91ad-610a-4ca5-b2f6-fae80e88bfc5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:41.968798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a91ad-610a-4ca5-b2f6-fae80e88bfc5 service glance] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:41.978373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1335/2634] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:15:41.980882 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2e567076c841ebb2c865e17ae25f6f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.981592 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2e567076c841ebb2c865e17ae25f6f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.982227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2e567076c841ebb2c865e17ae25f6f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:41.997107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1300/2635] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:41 2022] DELETE /identity/v3/users/5b2e567076c841ebb2c865e17ae25f6f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:42.002560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.002937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.003136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.025269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:42.026768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.046581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.048884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.102674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1336/2636] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:42.103288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:42.115384 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.117009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.129862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8593-59ca-4b54-acd5-ba64cd4f4f0c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.135134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.136253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.136538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.136822 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] PATH_INFO: `/v3/projects/0342793f5ac643a0802d85ceaaee8322` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.138877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0342793f5ac643a0802d85ceaaee8322)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.141877 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.156923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:42.187659 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e89ee-df22-4937-9b5a-8ee276008702 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.194776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0342793f5ac643a0802d85ceaaee8322', 'request_id': 'req-6bc82426-8d65-499e-934e-cc9ace0bb1ad'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.202439 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0342793f5ac643a0802d85ceaaee8322', 'request_id': 'req-6bc82426-8d65-499e-934e-cc9ace0bb1ad'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.208340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0342793f5ac643a0802d85ceaaee8322', 'request_id': 'req-6bc82426-8d65-499e-934e-cc9ace0bb1ad'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.222921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] The token cache is being invalidate because project 0342793f5ac643a0802d85ceaaee8322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:42.223245 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.223895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-6bc82426-8d65-499e-934e-cc9ace0bb1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.225790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1301/2637] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:42 2022] DELETE /identity/v3/projects/0342793f5ac643a0802d85ceaaee8322 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:42.230224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedf1a6-8dde-4af2-9eb9-5cf3fdc92cc8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.230520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedf1a6-8dde-4af2-9eb9-5cf3fdc92cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.230833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedf1a6-8dde-4af2-9eb9-5cf3fdc92cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.249599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.250360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.250676 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.250829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-0eedf1a6-8dde-4af2-9eb9-5cf3fdc92cc8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:42.251272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.252225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-0eedf1a6-8dde-4af2-9eb9-5cf3fdc92cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.252862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.324153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.325518 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.327102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1302/2638] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:42.336116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.336431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.336756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.344376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1337/2639] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:15:42.359510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:42.361622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.373033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1303/2640] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:42.376930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.390004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7cf93-1641-4f18-8d51-237a1beb1706 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.395674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.396391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.396942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.396942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.397672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:42.397980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.399648 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.417305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1338/2641] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:42.418758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.430880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5b537-01b5-4fe1-9b23-e57e8bf73013 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.436808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.437483 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.437827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.438117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.438844 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.441523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.474239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1304/2642] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:42.477994 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.490709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ca2f3-4dc2-4f18-b5f3-ec82422f16aa admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.575374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.576588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.576874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.577195 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.577973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.581055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.609436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1339/2643] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:42.613546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.624441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a2d7c-3311-4acb-9939-662f08832d94 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.629448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.630201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.630496 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.630828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.631603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.633250 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.648726 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1305/2644] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:42.652668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.663861 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dafec-2044-425c-9abe-134f1e149d90 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.664558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.666410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.668686 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.669354 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.669643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.669926 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.670728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.672335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.680751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef673e0e-b529-47df-aa95-8d0bfff0cd00 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.685629 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.686334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1340/2645] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:42.686795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.687083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.687213 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.687362 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] PATH_INFO: `/v3/projects/90e45f36590e48afa96451fd7ab2733e` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.688585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e45f36590e48afa96451fd7ab2733e)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.690548 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.693386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:42.698216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0101-d3e8-41fe-9b36-b773fdaf9c9d admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.706136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c5969b-12c9-4b22-9511-d79068a8fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.707087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5969b-12c9-4b22-9511-d79068a8fa94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.707956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5969b-12c9-4b22-9511-d79068a8fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.707956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5969b-12c9-4b22-9511-d79068a8fa94 admin admin] PATH_INFO: `/v3/projects/e93925dd72234e1e9917afcb1af25984/users/81b0095d8040427cb24e12797b3ba5fc/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.724465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5969b-12c9-4b22-9511-d79068a8fa94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93925dd72234e1e9917afcb1af25984, user_id=81b0095d8040427cb24e12797b3ba5fc, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.727217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c5969b-12c9-4b22-9511-d79068a8fa94 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.733512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:42.741018 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1341/2646] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:42 2022] PUT /identity/v3/projects/e93925dd72234e1e9917afcb1af25984/users/81b0095d8040427cb24e12797b3ba5fc/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:42.752282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e45f36590e48afa96451fd7ab2733e', 'request_id': 'req-bd50a864-a435-4ee1-96fb-07dd705e9dac'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.752750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.753052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.753343 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.760591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e45f36590e48afa96451fd7ab2733e', 'request_id': 'req-bd50a864-a435-4ee1-96fb-07dd705e9dac'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.766902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e45f36590e48afa96451fd7ab2733e', 'request_id': 'req-bd50a864-a435-4ee1-96fb-07dd705e9dac'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.782337 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] The token cache is being invalidate because project 90e45f36590e48afa96451fd7ab2733e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:42.782767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.784229 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.788369 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] MFA Rules not processed for user `81b0095d8040427cb24e12797b3ba5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:42.791682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1306/2647] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:42 2022] DELETE /identity/v3/projects/90e45f36590e48afa96451fd7ab2733e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:42.791930 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.792455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.793764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.856136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1342/2648] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:42.860214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.861823 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:42.863050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50a864-a435-4ee1-96fb-07dd705e9dac admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.925501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.926784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.927180 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.927463 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] PATH_INFO: `/v3/users/88d97c4001d84745965b48f302a1d4ff` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.928677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d97c4001d84745965b48f302a1d4ff)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.930634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.935820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb46e9a-4146-498b-8f58-f882421583b4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:42.941578 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:42.942442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:42.942442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:42.942693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:42.943359 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:42.945608 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:42.978685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1343/2649] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:42.980079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:42.982950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d97c4001d84745965b48f302a1d4ff'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:42.992344 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d97c4001d84745965b48f302a1d4ff'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.002284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:43.004771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d97c4001d84745965b48f302a1d4ff'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.005724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d97c4001d84745965b48f302a1d4ff'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.006838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d97c4001d84745965b48f302a1d4ff'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.022294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1307/2650] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:42 2022] DELETE /identity/v3/users/88d97c4001d84745965b48f302a1d4ff => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:43.041209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:43.053453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8c4d5-7019-4be0-a767-520440115fe6 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.126123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.128960 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.129235 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.129685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.130710 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.132834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.158218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1344/2651] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:42 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:43.162028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.163488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.177541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f5256-1ab4-4b8c-83a0-5728a7133587 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.183021 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.183542 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.183827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.184602 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.185403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.186982 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.200537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1308/2652] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:43.202327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.213606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa043-9cc8-4115-9733-2010f25ab578 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.219568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac448a09-b7dc-468e-84c2-2fe77fa513f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.220800 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac448a09-b7dc-468e-84c2-2fe77fa513f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.221092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac448a09-b7dc-468e-84c2-2fe77fa513f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.221404 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac448a09-b7dc-468e-84c2-2fe77fa513f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.222268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac448a09-b7dc-468e-84c2-2fe77fa513f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.223938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac448a09-b7dc-468e-84c2-2fe77fa513f6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.238660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1345/2653] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:43.240318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.250757 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc7ea0-57e8-4628-a461-1111f02b918c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.255596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.256573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.256866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.257144 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] PATH_INFO: `/v3/projects/1acf8a5fd1ec45918f79b82c65837b4d/users/99adfb2fa6344712a7e265084f5a07fc/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.279079 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1acf8a5fd1ec45918f79b82c65837b4d, user_id=99adfb2fa6344712a7e265084f5a07fc, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.281796 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.296632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1309/2654] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:43 2022] PUT /identity/v3/projects/1acf8a5fd1ec45918f79b82c65837b4d/users/99adfb2fa6344712a7e265084f5a07fc/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:43.308827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.309169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.309523 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.326301 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.328489 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.345005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] MFA Rules not processed for user `99adfb2fa6344712a7e265084f5a07fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:43.346911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.401833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318feb58-1c6a-4b81-b1bc-39fed6736f70 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.407550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.409102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.409386 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.409702 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] PATH_INFO: `/v3/projects/8acfcaf7e5cf4e0daddd4c32bff45616` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.411806 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8acfcaf7e5cf4e0daddd4c32bff45616)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.414073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.431367 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1346/2655] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:43.435745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.447604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685770d7-19b5-42d3-90a7-b2c5f1a6295e None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.452783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.453485 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.453773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.454046 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.455315 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.457267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.465902 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8acfcaf7e5cf4e0daddd4c32bff45616', 'request_id': 'req-c045a52c-e2ba-4e07-a027-53ede214ec15'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.472761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8acfcaf7e5cf4e0daddd4c32bff45616', 'request_id': 'req-c045a52c-e2ba-4e07-a027-53ede214ec15'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.479466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8acfcaf7e5cf4e0daddd4c32bff45616', 'request_id': 'req-c045a52c-e2ba-4e07-a027-53ede214ec15'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.488912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1347/2656] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:43.490027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.492633 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] The token cache is being invalidate because project 8acfcaf7e5cf4e0daddd4c32bff45616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:43.493007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.493774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-c045a52c-e2ba-4e07-a027-53ede214ec15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:43.495455 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1310/2657] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:43 2022] DELETE /identity/v3/projects/8acfcaf7e5cf4e0daddd4c32bff45616 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:43.501007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d5e5-cb91-429f-80d9-9f86bc38c769 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.501298 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d5e5-cb91-429f-80d9-9f86bc38c769 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.501623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d5e5-cb91-429f-80d9-9f86bc38c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.523094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c1f5d5e5-cb91-429f-80d9-9f86bc38c769 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:43.524328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c1f5d5e5-cb91-429f-80d9-9f86bc38c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.543409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:43.561853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:43.564793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc8ea-f5d5-4df5-8ccf-8ffc4e9c4156 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.571165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.572266 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.572590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.573216 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.574035 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.577074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.593718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1311/2658] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:43.600924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1348/2659] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:43.603530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012113-379d-428c-9a08-509c7410ef54 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.603945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012113-379d-428c-9a08-509c7410ef54 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.604415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59012113-379d-428c-9a08-509c7410ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.604758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.606561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.619314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3b6aa-c2d5-49d9-8d84-05d9f2b3a274 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.624544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.625198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.625617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.625902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.626009 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-59012113-379d-428c-9a08-509c7410ef54 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:43.626687 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.627221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-59012113-379d-428c-9a08-509c7410ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.628287 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.635925 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1312/2660] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:43.640160 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59012113-379d-428c-9a08-509c7410ef54 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.641884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1349/2661] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:43.646514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.649749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59012113-379d-428c-9a08-509c7410ef54 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.654466 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.655048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.655332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.655610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.656351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:43.656677 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.656844 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d4657-2d7e-49be-801d-44895a670137 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.658950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.662186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.663151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.663434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.663714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] PATH_INFO: `/v3/projects/27590a3aeea248c281e103a9f79a38dd/users/b58df3a8f4374aefa3e2e1091c08611e/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.680773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27590a3aeea248c281e103a9f79a38dd, user_id=b58df3a8f4374aefa3e2e1091c08611e, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.682645 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1313/2662] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:15:43.683699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.686875 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.699402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08745b7-5139-430f-8a80-46a276bd4878 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.699883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1350/2663] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:43 2022] PUT /identity/v3/projects/27590a3aeea248c281e103a9f79a38dd/users/b58df3a8f4374aefa3e2e1091c08611e/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:43.706654 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.760333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.761137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.761427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.761705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.762397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.764509 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.772543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f465-07af-4932-87ba-9b46f5ab0637 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.779887 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.780707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.781066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.781416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.782232 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.784156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.798053 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1314/2664] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:43.798759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1351/2665] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:43.799507 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.799657 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.809650 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2e98-3dd6-4828-9a57-cbc320c5ec6f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.864308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e617ea-c0a6-4fdc-b9f1-702fca816023 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.867576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad6b85-332d-4a4a-a5a8-d3a748e67b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.868524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad6b85-332d-4a4a-a5a8-d3a748e67b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.868865 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad6b85-332d-4a4a-a5a8-d3a748e67b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.868974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.869466 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad6b85-332d-4a4a-a5a8-d3a748e67b86 admin admin] PATH_INFO: `/v3/projects/27590a3aeea248c281e103a9f79a38dd/users/b58df3a8f4374aefa3e2e1091c08611e/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.869672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.869907 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.870237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.870968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.871571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad6b85-332d-4a4a-a5a8-d3a748e67b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27590a3aeea248c281e103a9f79a38dd, user_id=b58df3a8f4374aefa3e2e1091c08611e, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.872856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.874256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad6b85-332d-4a4a-a5a8-d3a748e67b86 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.887041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1352/2666] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:43 2022] PUT /identity/v3/projects/27590a3aeea248c281e103a9f79a38dd/users/b58df3a8f4374aefa3e2e1091c08611e/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:43.895200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1315/2667] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:43.899105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.899430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.899723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.900005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.909438 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c18b68-ad35-42ff-b323-d4541db64bf7 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:43.927851 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] MFA Rules not processed for user `b58df3a8f4374aefa3e2e1091c08611e`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:43.929081 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:43.944201 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:43.948991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:43.970963 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:43.971992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:43.972281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:43.972585 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:43.973324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:43.974968 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:43.985347 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1316/2668] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:15:43.989164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:43.999132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9c186-2b79-4425-af6b-485ef27e9bf0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.004452 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.005108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.005394 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.005673 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.006401 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.008036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.009604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1353/2669] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:44.020470 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1317/2670] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:44.021694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.023137 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.023497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.033350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a78ee-a4e6-44f0-b111-6cfffd742b68 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.038501 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee34817-e567-4afb-96ac-9930aaced9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.039421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee34817-e567-4afb-96ac-9930aaced9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.039708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee34817-e567-4afb-96ac-9930aaced9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.039984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee34817-e567-4afb-96ac-9930aaced9f9 admin admin] PATH_INFO: `/v3/projects/61ee2421192d4001bc2706a9d26d4a37/users/0ff9dd2129844a18a2048a7f5b629270/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.056115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee34817-e567-4afb-96ac-9930aaced9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ee2421192d4001bc2706a9d26d4a37, user_id=0ff9dd2129844a18a2048a7f5b629270, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.058955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee34817-e567-4afb-96ac-9930aaced9f9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.074370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1318/2671] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:44 2022] PUT /identity/v3/projects/61ee2421192d4001bc2706a9d26d4a37/users/0ff9dd2129844a18a2048a7f5b629270/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:15:44.085068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.085279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.085444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9876195-1be6-448a-9ff2-a6333c817ab7 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.085590 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.112022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] MFA Rules not processed for user `0ff9dd2129844a18a2048a7f5b629270`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:44.113276 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.138070 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.138992 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.139109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.139468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.192707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1319/2672] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:44.204016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.205577 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.206083 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.207249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.217741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1354/2673] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:44.221007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11ebb9-dd57-46ed-a6ea-d21aba1f82c8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.224974 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a1ac7-7064-4f46-9369-9523615993f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.225775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a1ac7-7064-4f46-9369-9523615993f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.226101 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a1ac7-7064-4f46-9369-9523615993f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.226382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a1ac7-7064-4f46-9369-9523615993f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.235678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a1ac7-7064-4f46-9369-9523615993f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.237164 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a1ac7-7064-4f46-9369-9523615993f8 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.246349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1320/2674] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:15:44.266080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.294125 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=100 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:44.339373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cacff9-f395-4bc5-9f69-20e49db304d5 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.344381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.346054 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.346425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.346780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] PATH_INFO: `/v3/users/0ff9dd2129844a18a2048a7f5b629270` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.347919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff9dd2129844a18a2048a7f5b629270)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.349845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.411750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff9dd2129844a18a2048a7f5b629270'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:44.419565 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff9dd2129844a18a2048a7f5b629270'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:44.433463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff9dd2129844a18a2048a7f5b629270'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:44.434286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff9dd2129844a18a2048a7f5b629270'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:44.435037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff9dd2129844a18a2048a7f5b629270'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:44.450390 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1355/2675] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:44 2022] DELETE /identity/v3/users/0ff9dd2129844a18a2048a7f5b629270 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:44.455730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.456225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.456351 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.474423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:44.475768 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.542348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1321/2676] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:44.552560 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.554283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.621866 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf82ffd-ff98-4524-ba38-51f59bb8502b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.626290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.627488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.627767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.628042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.646672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.646672 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.661453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1356/2677] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:44.767684 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.769102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.785574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:44.840364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec631017-a486-4f54-99eb-514f0893dd6e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.843954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.844692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.844936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.845315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.847720 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.909908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.911391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e69b-8aca-4729-9ccd-584ef9507053 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:44.921939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1322/2678] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:44.958549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:44.960819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:44.976254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a0eb-b34b-4048-a68c-de9608217cd4 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:44.984590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:44.985503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:44.986434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:44.986758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] PATH_INFO: `/v3/projects/61ee2421192d4001bc2706a9d26d4a37` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:44.988079 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ee2421192d4001bc2706a9d26d4a37)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:44.990080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.040679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:45.050470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ee2421192d4001bc2706a9d26d4a37', 'request_id': 'req-f9cac096-8d86-4ff3-a3fd-5d01a2425641'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:45.059053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ee2421192d4001bc2706a9d26d4a37', 'request_id': 'req-f9cac096-8d86-4ff3-a3fd-5d01a2425641'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:45.065133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ee2421192d4001bc2706a9d26d4a37', 'request_id': 'req-f9cac096-8d86-4ff3-a3fd-5d01a2425641'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:45.080441 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] The token cache is being invalidate because project 61ee2421192d4001bc2706a9d26d4a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:15:45.080724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:45.081344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f9cac096-8d86-4ff3-a3fd-5d01a2425641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:45.082911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1357/2679] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:15:44 2022] DELETE /identity/v3/projects/61ee2421192d4001bc2706a9d26d4a37 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:45.088132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.088541 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.088892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.111546 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:45.112749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:45.179895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1323/2680] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:15:45.187681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.188139 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.188139 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.211044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:45.212381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:45.222155 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1358/2681] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:45.227388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.238839 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a715c38-a28c-4bda-b80d-a6a1834b64fa None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.243996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.244669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.244956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.245237 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.245983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:15:45.246291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.249464 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.266143 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1324/2682] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:15:45.270809 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.282442 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3d192-433e-45d9-84cd-903b9f3a3de0 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.289936 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.290750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.291031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.291307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.292257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.294219 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.322651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1359/2683] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:15:45 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:15:45.325060 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.336165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67decbd-cd53-468d-9d40-0b5e3bfbe51d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.386678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:45.393511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.394171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.394451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.394753 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.395434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.397318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.416954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1325/2684] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:15:45 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:15:45.421004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.431684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb66ac0-73f6-41e6-bbf4-819dc47301a4 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.436209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.436854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.437135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.437405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.438119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.439737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.449759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1360/2685] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:45.453578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.463674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d80a8d-4930-4b60-b041-914ee491ade0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.468370 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566a1c8-3b46-4405-a33b-4911ca32061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.469013 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566a1c8-3b46-4405-a33b-4911ca32061d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.469295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566a1c8-3b46-4405-a33b-4911ca32061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.469568 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566a1c8-3b46-4405-a33b-4911ca32061d admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.470811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566a1c8-3b46-4405-a33b-4911ca32061d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.472382 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566a1c8-3b46-4405-a33b-4911ca32061d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.482187 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1326/2686] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:15:45.486918 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.496963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3488f-b1ba-4ca7-9124-63b58ed94fc1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.501566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.502430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.502737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.503013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] PATH_INFO: `/v3/projects/5bac5f1416384fab979f884456fe8d41/users/9315276e2b0a4a5ea5e207aab55ccc6c/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.517895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bac5f1416384fab979f884456fe8d41, user_id=9315276e2b0a4a5ea5e207aab55ccc6c, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.520590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.534285 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1361/2687] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:15:45 2022] PUT /identity/v3/projects/5bac5f1416384fab979f884456fe8d41/users/9315276e2b0a4a5ea5e207aab55ccc6c/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:45.545289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.545576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.545856 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.573198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] MFA Rules not processed for user `9315276e2b0a4a5ea5e207aab55ccc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:45.574408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:45.640305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1327/2688] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:15:45.652911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:45.654638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:45.716411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2109f-245c-40c0-a911-90bea44da354 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:45.720029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:45.720822 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:45.721886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:45.722138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:45.730920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:45.732361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:45.741550 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1362/2689] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:54.178610 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:54.180082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:54.191838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c77ce8-fd26-42ff-af6d-cab071d6b7b2 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:54.195403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:54.196047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:54.196330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:54.196604 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:54.198335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:54.257445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:54.258904 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460333a-2f81-4b09-803f-1456a8a6336d service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:54.268065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1328/2690] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:15:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:15:55.741874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:15:59.777703 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:15:59.779552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:15:59.857045 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbdc6f-057b-4a4f-9074-94a00f5c7c6d service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:15:59.861498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:15:59.862389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:59.862697 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:59.862973 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] PATH_INFO: `/v3/users/81b0095d8040427cb24e12797b3ba5fc` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:59.864167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b0095d8040427cb24e12797b3ba5fc)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:15:59.866667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:15:59.911146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b0095d8040427cb24e12797b3ba5fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:59.918071 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b0095d8040427cb24e12797b3ba5fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:59.930473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b0095d8040427cb24e12797b3ba5fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:59.931142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b0095d8040427cb24e12797b3ba5fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:59.931741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b0095d8040427cb24e12797b3ba5fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:15:59.944398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1363/2691] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:15:59 2022] DELETE /identity/v3/users/81b0095d8040427cb24e12797b3ba5fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:15:59.948918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:15:59.949371 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:15:59.949510 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:15:59.966689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:15:59.967868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:00.027916 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1329/2692] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:16:00.037679 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:00.039111 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:00.097874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358f63-0d19-43ee-82e8-e14e293bfe98 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:00.102357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:00.103320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:00.103620 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:00.103902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:00.113788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:00.115810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:00.125270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1364/2693] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:00.397985 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:00.399547 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:00.406664 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:00.414138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec8533-200c-4818-ae32-f84a86c3c0a0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:00.422868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:00.423657 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:00.423939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:00.424218 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] PATH_INFO: `/v3/projects/e93925dd72234e1e9917afcb1af25984` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:00.425407 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93925dd72234e1e9917afcb1af25984)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:00.427477 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:00.478367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93925dd72234e1e9917afcb1af25984', 'request_id': 'req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.484468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93925dd72234e1e9917afcb1af25984', 'request_id': 'req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.490429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93925dd72234e1e9917afcb1af25984', 'request_id': 'req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.504097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] The token cache is being invalidate because project e93925dd72234e1e9917afcb1af25984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:00.504388 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.505023 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.507194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1330/2694] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:00 2022] DELETE /identity/v3/projects/e93925dd72234e1e9917afcb1af25984 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:00.511446 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:00.521978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a988-8ee8-46c2-8890-c1f8597eb5cb service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:00.539977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:00.580890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:00.581764 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:00.582050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:00.582328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] PATH_INFO: `/v3/users/99adfb2fa6344712a7e265084f5a07fc` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:00.583576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99adfb2fa6344712a7e265084f5a07fc)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:00.585515 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:00.635043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99adfb2fa6344712a7e265084f5a07fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.642912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99adfb2fa6344712a7e265084f5a07fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.657328 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99adfb2fa6344712a7e265084f5a07fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.658003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99adfb2fa6344712a7e265084f5a07fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.658724 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99adfb2fa6344712a7e265084f5a07fc'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:00.671147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1365/2695] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:00 2022] DELETE /identity/v3/users/99adfb2fa6344712a7e265084f5a07fc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:00.952365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:00.953930 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:00.972357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2e79-a9b7-43be-a6a5-7db4f6626d5f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:01.038936 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:01.039981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.040263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.040544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] PATH_INFO: `/v3/projects/1acf8a5fd1ec45918f79b82c65837b4d` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.041752 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1acf8a5fd1ec45918f79b82c65837b4d)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:01.043783 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:01.089823 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1acf8a5fd1ec45918f79b82c65837b4d', 'request_id': 'req-78acafe9-263c-484a-a4f9-0f638a6e4b36'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.096708 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1acf8a5fd1ec45918f79b82c65837b4d', 'request_id': 'req-78acafe9-263c-484a-a4f9-0f638a6e4b36'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.102957 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1acf8a5fd1ec45918f79b82c65837b4d', 'request_id': 'req-78acafe9-263c-484a-a4f9-0f638a6e4b36'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.117600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] The token cache is being invalidate because project 1acf8a5fd1ec45918f79b82c65837b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:01.117892 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.118553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.120139 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1331/2696] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:00 2022] DELETE /identity/v3/projects/1acf8a5fd1ec45918f79b82c65837b4d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:01.124393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:01.125874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:01.140037 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd790-208b-40e5-8f4d-2a3f3d70a9c7 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:01.199016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:01.199896 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.200172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.200443 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] PATH_INFO: `/v3/users/b58df3a8f4374aefa3e2e1091c08611e` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.201641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58df3a8f4374aefa3e2e1091c08611e)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:01.203568 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:01.248228 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:01.251198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58df3a8f4374aefa3e2e1091c08611e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.258951 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58df3a8f4374aefa3e2e1091c08611e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.272107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58df3a8f4374aefa3e2e1091c08611e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.272779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58df3a8f4374aefa3e2e1091c08611e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.273368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-f374b1eb-a915-4a94-9831-46982a74b5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58df3a8f4374aefa3e2e1091c08611e'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.285516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1366/2697] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:01 2022] DELETE /identity/v3/users/b58df3a8f4374aefa3e2e1091c08611e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:01.554295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:01.555939 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:01.569346 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acafe9-263c-484a-a4f9-0f638a6e4b36 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:01.581724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:01.628415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:01.629273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.629549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.629816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] PATH_INFO: `/v3/projects/27590a3aeea248c281e103a9f79a38dd` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.631048 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27590a3aeea248c281e103a9f79a38dd)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:01.632947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:01.679263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27590a3aeea248c281e103a9f79a38dd', 'request_id': 'req-b2b54d40-2d70-45fb-a341-8703f46b8a56'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.685265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27590a3aeea248c281e103a9f79a38dd', 'request_id': 'req-b2b54d40-2d70-45fb-a341-8703f46b8a56'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.690824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27590a3aeea248c281e103a9f79a38dd', 'request_id': 'req-b2b54d40-2d70-45fb-a341-8703f46b8a56'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.702721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] The token cache is being invalidate because project 27590a3aeea248c281e103a9f79a38dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:01.703012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.703637 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-b2b54d40-2d70-45fb-a341-8703f46b8a56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:01.705170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1332/2698] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:01 2022] DELETE /identity/v3/projects/27590a3aeea248c281e103a9f79a38dd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:01.711935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.712225 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.712506 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.730377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:01.731643 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:01.790572 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1367/2699] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:01.798931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.799226 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.799508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.818183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:01.819423 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:01.827397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1333/2700] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:01.831525 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:01.841171 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de461d65-45ae-4c90-84a0-081638a6da88 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:01.845752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:01.846429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.846743 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.847022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.847772 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:16:01.848077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:01.850163 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:01.861436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1368/2701] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:16:01.866077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:01.875183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fa3f1-0819-49bc-bb68-b9fb7e72cdcb None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:01.879847 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:01.880506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.880795 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.881078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.881781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:01.883715 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:01.908920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1334/2702] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:01 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:01.913531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:01.923415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c801f-6ca4-4401-8ef9-a581bb046265 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:01.990269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:01.990979 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:01.991268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:01.991548 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:01.992253 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:01.994196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.017963 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1369/2703] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:01 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:02.022308 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.036077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ff8e1-4cf3-4771-ba59-c28336b56469 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.041704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.042756 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.043188 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.043469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.044200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.045776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.062687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1335/2704] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:02.064712 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.075592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8f1ab-2643-4a50-a361-bbb2f8b80186 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.080435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934ecaf-7533-41c5-bbf5-a6adefdd039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.081204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ecaf-7533-41c5-bbf5-a6adefdd039a admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.081562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ecaf-7533-41c5-bbf5-a6adefdd039a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.081911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934ecaf-7533-41c5-bbf5-a6adefdd039a admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.082754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934ecaf-7533-41c5-bbf5-a6adefdd039a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.084727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934ecaf-7533-41c5-bbf5-a6adefdd039a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.095919 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1370/2705] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:02.100299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.110134 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff15ea-674e-47a3-87a8-e9edd8fec976 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.115305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.116182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.116478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.116761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] PATH_INFO: `/v3/projects/adefa06ee0e54df587d4f7d1b606e367/users/d0feea61972f42c7a2e2b1e1a4dda03d/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.132741 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=adefa06ee0e54df587d4f7d1b606e367, user_id=d0feea61972f42c7a2e2b1e1a4dda03d, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.135821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.149825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1336/2706] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:02 2022] PUT /identity/v3/projects/adefa06ee0e54df587d4f7d1b606e367/users/d0feea61972f42c7a2e2b1e1a4dda03d/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:02.161856 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.162385 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.162598 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.190633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] MFA Rules not processed for user `d0feea61972f42c7a2e2b1e1a4dda03d`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:02.192141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:02.213981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:02.276373 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1371/2707] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:02.284532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.333850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:02.355243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdb98-4dae-4f3f-801b-c075f9cd8b9a admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.359986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.360681 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.360966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.361246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.361955 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.363897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.389090 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1337/2708] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:02 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:16:02.393123 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.403708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ebf23-b43d-4d54-b3b5-ff424d080307 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.460198 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.460871 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.461160 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.461440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.462142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.464101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.488138 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1372/2709] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:02 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:02.490222 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.500555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd5c2-8d3a-4ffe-8e0f-3049f6dc54c4 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.505204 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.505863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.506148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.506429 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.507190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.508777 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.519380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1338/2710] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:02.523069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.532971 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec3dc5-1899-4522-bb26-5e5e396ad51e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.538150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98dd3a0-da4d-4f24-b833-309ebfdb8648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.538847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd3a0-da4d-4f24-b833-309ebfdb8648 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.539156 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd3a0-da4d-4f24-b833-309ebfdb8648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.539462 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd3a0-da4d-4f24-b833-309ebfdb8648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.540546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dd3a0-da4d-4f24-b833-309ebfdb8648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.542132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98dd3a0-da4d-4f24-b833-309ebfdb8648 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.552778 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1373/2711] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:02.556639 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.566390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847a8e-9728-4a8c-ae77-da16af9f2dfd admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.571003 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.571866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.572149 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.572424 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] PATH_INFO: `/v3/projects/5b459c89aa654afcb3175c8f8be74284/users/9ce770b327ab48c6a53ece44cdc91089/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.586699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b459c89aa654afcb3175c8f8be74284, user_id=9ce770b327ab48c6a53ece44cdc91089, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.589373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.602829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1339/2712] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:02 2022] PUT /identity/v3/projects/5b459c89aa654afcb3175c8f8be74284/users/9ce770b327ab48c6a53ece44cdc91089/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:02.613003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.613281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.613562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.639876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] MFA Rules not processed for user `9ce770b327ab48c6a53ece44cdc91089`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:02.641227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:02.712470 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1374/2713] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:02.716867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.777246 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50dad2-435b-4817-b5bc-f255ef93db48 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.782119 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.782818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.783110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.783392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.784087 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.786020 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.811368 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1340/2714] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:02 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:02.815399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.825807 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75eea4-788c-4959-9fc7-e70ceb2fc507 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.845935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:02.887494 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.888176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.888464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.888741 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.889447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.891416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.914472 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1375/2715] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:02 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:02.918631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.929534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6c96-3a28-49a8-90d1-0db104e04937 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.934538 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.935238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.935530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.935855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.936699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.938649 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:02.953035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1341/2716] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:02.954275 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:02.965014 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f9503-feac-4b7c-820c-6fe817cb8d3c admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:02.969817 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:02.970870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:02.971158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:02.971440 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] PATH_INFO: `/v3/projects/88aad37973eb4942b2649394973aacc5/users/893782c3db8d4ea2961b5b7a38f4f8f3/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:02.987066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88aad37973eb4942b2649394973aacc5, user_id=893782c3db8d4ea2961b5b7a38f4f8f3, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:02.989750 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:03.003641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1376/2717] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:02 2022] PUT /identity/v3/projects/88aad37973eb4942b2649394973aacc5/users/893782c3db8d4ea2961b5b7a38f4f8f3/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:03.007479 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:03.017703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d2e97-9f8b-4f76-9ed0-445d5edd239b admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:03.079243 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87598c94-fdea-497a-aead-fb42982e26e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:03.079919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87598c94-fdea-497a-aead-fb42982e26e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:03.080200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87598c94-fdea-497a-aead-fb42982e26e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:03.080474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87598c94-fdea-497a-aead-fb42982e26e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:03.081206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87598c94-fdea-497a-aead-fb42982e26e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:03.082831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87598c94-fdea-497a-aead-fb42982e26e6 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:03.097816 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1342/2718] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:03.099341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:03.109911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d41d1d-c430-481b-b5c0-78a6bbec0ca1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:03.117802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:03.119153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:03.119475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:03.119831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] PATH_INFO: `/v3/projects/88aad37973eb4942b2649394973aacc5/users/893782c3db8d4ea2961b5b7a38f4f8f3/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:03.122329 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88aad37973eb4942b2649394973aacc5, user_id=893782c3db8d4ea2961b5b7a38f4f8f3, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:03.125767 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:03.137610 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1377/2719] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:03 2022] PUT /identity/v3/projects/88aad37973eb4942b2649394973aacc5/users/893782c3db8d4ea2961b5b7a38f4f8f3/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:03.149253 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:03.149576 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:03.149941 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:03.156197 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:03.177799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] MFA Rules not processed for user `893782c3db8d4ea2961b5b7a38f4f8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:03.179041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:03.244375 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1343/2720] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:03.256020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:03.257757 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:03.315945 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d1af5-52e8-4c9a-b18b-19dc6c5d28b1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:03.320186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:03.320831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:03.321114 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:03.321416 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:03.330838 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:03.332368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:03.343109 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1378/2721] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:03.383776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:03.395660 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5744dcc-f62b-4af2-a41a-6c6a963a0027 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:03.400177 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a8469-34ec-4a2c-b357-c4db9d8a7d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:03.400838 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a8469-34ec-4a2c-b357-c4db9d8a7d05 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:03.401432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a8469-34ec-4a2c-b357-c4db9d8a7d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:03.401686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a8469-34ec-4a2c-b357-c4db9d8a7d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:03.462408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a8469-34ec-4a2c-b357-c4db9d8a7d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:03.463995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a8469-34ec-4a2c-b357-c4db9d8a7d05 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:03.474928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1344/2722] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:12.580993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:12.646662 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16380f-41c7-4f40-b327-bb92ecb5fc64 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:12.651290 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:12.652152 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:12.652434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:12.652707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] PATH_INFO: `/v3/users/d0feea61972f42c7a2e2b1e1a4dda03d` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:12.653884 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0feea61972f42c7a2e2b1e1a4dda03d)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:12.655820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:12.676806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:12.709230 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0feea61972f42c7a2e2b1e1a4dda03d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:12.716683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0feea61972f42c7a2e2b1e1a4dda03d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:12.728738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0feea61972f42c7a2e2b1e1a4dda03d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:12.729377 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0feea61972f42c7a2e2b1e1a4dda03d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:12.729984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0feea61972f42c7a2e2b1e1a4dda03d'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:12.742674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1379/2723] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:12 2022] DELETE /identity/v3/users/d0feea61972f42c7a2e2b1e1a4dda03d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:12.747437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:12.747927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:12.747927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:12.766360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:12.767615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:12.826292 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1345/2724] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:12.835649 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:12.837028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:12.893902 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aaece-e29c-4364-9eab-e46526ce6952 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:12.897512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:12.898150 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:12.898431 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:12.898737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:12.907429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:12.908847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:12.918520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1380/2725] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:13.182094 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:13.183927 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:13.198258 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad47cf9-ee1d-4821-ab5b-69ca781f8791 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:13.204153 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:13.205033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.205322 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.205603 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] PATH_INFO: `/v3/projects/adefa06ee0e54df587d4f7d1b606e367` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.206859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=adefa06ee0e54df587d4f7d1b606e367)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:13.208732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:13.232780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:13.255825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adefa06ee0e54df587d4f7d1b606e367', 'request_id': 'req-d188a084-a919-42c6-bdba-7c2d0ac82a9d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.262271 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adefa06ee0e54df587d4f7d1b606e367', 'request_id': 'req-d188a084-a919-42c6-bdba-7c2d0ac82a9d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.268461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adefa06ee0e54df587d4f7d1b606e367', 'request_id': 'req-d188a084-a919-42c6-bdba-7c2d0ac82a9d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.281961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] The token cache is being invalidate because project adefa06ee0e54df587d4f7d1b606e367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:13.282248 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.282900 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.284427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1346/2726] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:13 2022] DELETE /identity/v3/projects/adefa06ee0e54df587d4f7d1b606e367 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:13.288512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:13.298788 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189079a-eaab-40e0-8d2b-6407ce1bc29b service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:13.354658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:13.355551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.355831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.356101 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] PATH_INFO: `/v3/users/9ce770b327ab48c6a53ece44cdc91089` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.357281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce770b327ab48c6a53ece44cdc91089)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:13.359242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:13.407244 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:13.409566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce770b327ab48c6a53ece44cdc91089'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.419117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce770b327ab48c6a53ece44cdc91089'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.433493 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce770b327ab48c6a53ece44cdc91089'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.434186 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce770b327ab48c6a53ece44cdc91089'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.434937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-db9606f5-ff61-4e5f-a7bd-0851bd4e9845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce770b327ab48c6a53ece44cdc91089'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.449733 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1381/2727] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:13 2022] DELETE /identity/v3/users/9ce770b327ab48c6a53ece44cdc91089 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:13.514983 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:13.516932 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:13.533268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188a084-a919-42c6-bdba-7c2d0ac82a9d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:13.593647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:13.594529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.594864 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.595133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] PATH_INFO: `/v3/users/9315276e2b0a4a5ea5e207aab55ccc6c` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.596326 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9315276e2b0a4a5ea5e207aab55ccc6c)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:13.598415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:13.648972 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9315276e2b0a4a5ea5e207aab55ccc6c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.657954 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9315276e2b0a4a5ea5e207aab55ccc6c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.670803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9315276e2b0a4a5ea5e207aab55ccc6c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.671456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9315276e2b0a4a5ea5e207aab55ccc6c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.672055 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9315276e2b0a4a5ea5e207aab55ccc6c'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.684217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1347/2728] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:13 2022] DELETE /identity/v3/users/9315276e2b0a4a5ea5e207aab55ccc6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:13.688701 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.689222 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.689332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.707883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:13.709124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:13.738591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:13.740116 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:13.774717 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:13.779949 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1382/2729] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:13.789623 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:13.791145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:13.811273 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e153c1-d553-48f0-aa02-444b094f6cd3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:13.815860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:13.816724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.817005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.817285 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] PATH_INFO: `/v3/projects/5b459c89aa654afcb3175c8f8be74284` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.818472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b459c89aa654afcb3175c8f8be74284)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:13.820911 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:13.847839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d292d4-5d2a-4e7e-9556-20cc9ddb4f6b None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:13.851536 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:13.852189 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.852476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.852755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.862129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:13.863595 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:13.867918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b459c89aa654afcb3175c8f8be74284', 'request_id': 'req-517807c7-84db-4bff-a831-41f4715affcf'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.873387 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1383/2730] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:13.874072 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b459c89aa654afcb3175c8f8be74284', 'request_id': 'req-517807c7-84db-4bff-a831-41f4715affcf'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.880352 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b459c89aa654afcb3175c8f8be74284', 'request_id': 'req-517807c7-84db-4bff-a831-41f4715affcf'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.893348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] The token cache is being invalidate because project 5b459c89aa654afcb3175c8f8be74284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:13.893635 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.894279 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:13.896022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1348/2731] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:13 2022] DELETE /identity/v3/projects/5b459c89aa654afcb3175c8f8be74284 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:13.900437 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:13.909983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2038a-0a91-436a-84bb-98de14e05e8b service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:13.971292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:13.972181 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:13.972455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:13.972729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] PATH_INFO: `/v3/users/893782c3db8d4ea2961b5b7a38f4f8f3` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:13.974031 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893782c3db8d4ea2961b5b7a38f4f8f3)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:13.975989 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:13.987903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:14.059843 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893782c3db8d4ea2961b5b7a38f4f8f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.075773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893782c3db8d4ea2961b5b7a38f4f8f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.091665 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893782c3db8d4ea2961b5b7a38f4f8f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.092633 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893782c3db8d4ea2961b5b7a38f4f8f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.093417 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-39d299ac-3ab6-4a68-af47-fc987fcf51e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893782c3db8d4ea2961b5b7a38f4f8f3'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.112048 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1384/2732] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:13 2022] DELETE /identity/v3/users/893782c3db8d4ea2961b5b7a38f4f8f3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:14.163460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.168725 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.185543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517807c7-84db-4bff-a831-41f4715affcf admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.282643 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.283676 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.284068 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.284814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] PATH_INFO: `/v3/projects/5bac5f1416384fab979f884456fe8d41` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.286305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bac5f1416384fab979f884456fe8d41)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.288410 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.349705 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bac5f1416384fab979f884456fe8d41', 'request_id': 'req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.359380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bac5f1416384fab979f884456fe8d41', 'request_id': 'req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.367694 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:14.378967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bac5f1416384fab979f884456fe8d41', 'request_id': 'req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.397651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] The token cache is being invalidate because project 5bac5f1416384fab979f884456fe8d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:14.398077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.398929 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.402430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1349/2733] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:14 2022] DELETE /identity/v3/projects/5bac5f1416384fab979f884456fe8d41 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:14.406286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8dc5-afd2-49e3-9ead-1d6d978da889 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.406841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8dc5-afd2-49e3-9ead-1d6d978da889 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.407116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f8dc5-afd2-49e3-9ead-1d6d978da889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.427606 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-751f8dc5-afd2-49e3-9ead-1d6d978da889 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:14.428797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-751f8dc5-afd2-49e3-9ead-1d6d978da889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.446672 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.448575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.500026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1385/2734] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:14.508380 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.508658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.508943 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.521369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388cbca-50d7-4e1a-a9a2-38ed22c0b2a3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.527217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.528076 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.528360 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.528634 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] PATH_INFO: `/v3/projects/88aad37973eb4942b2649394973aacc5` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.529690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:14.529825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] RBAC: Authorizing `identity:delete_project(project_id=88aad37973eb4942b2649394973aacc5)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.530955 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.531737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.538827 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1386/2735] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:16:14.542982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.552209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae764b-15e0-4c09-9537-5000e786600c None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.557003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.557642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.557925 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.558201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.559199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:16:14.559602 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.561337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.578895 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1387/2736] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:16:14.580159 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.581588 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88aad37973eb4942b2649394973aacc5', 'request_id': 'req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.588566 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88aad37973eb4942b2649394973aacc5', 'request_id': 'req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.591362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327da089-ef2f-4344-a34f-dacb86966fa9 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.593999 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88aad37973eb4942b2649394973aacc5', 'request_id': 'req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.608321 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] The token cache is being invalidate because project 88aad37973eb4942b2649394973aacc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:14.608611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.609247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-7dfeea8b-5c24-4f24-952e-4ab1efb6245b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:14.611227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1350/2737] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:14 2022] DELETE /identity/v3/projects/88aad37973eb4942b2649394973aacc5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:14.616967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a9ee6-7102-4c29-9f14-aab2dee1b16b None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.617380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a9ee6-7102-4c29-9f14-aab2dee1b16b None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.617820 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a9ee6-7102-4c29-9f14-aab2dee1b16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.639899 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-740a9ee6-7102-4c29-9f14-aab2dee1b16b None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:14.641206 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-740a9ee6-7102-4c29-9f14-aab2dee1b16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.655344 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.656011 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.656292 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.656566 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.657998 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.660937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.669828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:14.692473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1388/2738] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:14.696999 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.698705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.707041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1351/2739] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:16:14.719764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.719764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.719764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.738378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:14.739611 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.752534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:14.763508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4261f4a-e8bb-458b-ab45-f44fb741ac2b admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.768085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.768760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.769047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.769326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.770023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.771978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.793883 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1389/2740] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:16:14.797890 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.807772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1352/2741] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:16:14.808279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968ab11-5470-4ced-9481-d06060da47fd admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.811912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.812688 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.813342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.813624 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.813900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.814640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.816313 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.821216 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2f39c-8b77-4e59-8979-80ae5863a669 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.825836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.826480 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.826815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.826914 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1390/2742] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:16:14.829990 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.830774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.830905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:16:14.831147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.832811 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.841052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7a3f5-db2b-4809-bcd9-a47b22bfe235 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.847737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1353/2743] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:16:14.848857 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.849421 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.850105 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.850389 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.850685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.851510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.853087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.858453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669738c-61e5-4a32-abe6-b4e376c48c08 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.863050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.863510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1391/2744] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:16:14.863730 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.864015 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.864291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.864967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.867458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.867789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.877196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba762-7d89-425e-a609-b3b571a9a187 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:14.881789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c58dd-022b-45e5-886e-2ce1dfeef9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:14.883204 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c58dd-022b-45e5-886e-2ce1dfeef9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.883474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c58dd-022b-45e5-886e-2ce1dfeef9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.883819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c58dd-022b-45e5-886e-2ce1dfeef9ff admin admin] PATH_INFO: `/v3/projects/908836d993404006970ed39a9613b600/users/96d247f90aa5449dbd54a4ea86458f17/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.893800 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1354/2745] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:14.897685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:14.900870 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c58dd-022b-45e5-886e-2ce1dfeef9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=908836d993404006970ed39a9613b600, user_id=96d247f90aa5449dbd54a4ea86458f17, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:14.903569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c58dd-022b-45e5-886e-2ce1dfeef9ff admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:14.916755 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1392/2746] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:14 2022] PUT /identity/v3/projects/908836d993404006970ed39a9613b600/users/96d247f90aa5449dbd54a4ea86458f17/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:14.927855 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:14.928145 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:14.928426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:14.955916 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] MFA Rules not processed for user `96d247f90aa5449dbd54a4ea86458f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:14.957110 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:14.963297 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495f2c6-d912-4632-b69d-949536f0f952 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.019731 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.020409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.020699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.020977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.021670 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.021824 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1393/2747] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:16:15.023692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.032474 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.033920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:15.047855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1355/2748] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:14 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:16:15.048867 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.059031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c54f53a-bb02-4c93-bbc0-d0f6cd06a771 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.063937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.064492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:15.064627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.064869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.065154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.065966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.067615 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.077947 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1356/2749] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:16:15.082236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.091704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c5e4f-09db-473c-9e05-a7d4e91b5a13 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.095950 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8d10d-5869-4c7c-a55c-d16776b17527 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.096415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.097085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.097372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.097661 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.098377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.099677 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf2646-4827-4cbc-ad0b-ed204dcda0d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.100039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.100323 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf2646-4827-4cbc-ad0b-ed204dcda0d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.100611 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf2646-4827-4cbc-ad0b-ed204dcda0d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.100889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf2646-4827-4cbc-ad0b-ed204dcda0d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.105846 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:15.110435 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf2646-4827-4cbc-ad0b-ed204dcda0d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.111889 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf2646-4827-4cbc-ad0b-ed204dcda0d8 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.112855 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1357/2750] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:15.116474 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.122707 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1394/2751] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:16:15.127128 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8547c-d60e-4cb0-85ce-4abcca91d5fc admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.131688 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.132591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.132876 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.133161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] PATH_INFO: `/v3/projects/d60b555b62194d01a135bfff6e907570/users/cd6bd87de8a745eb8e097bbf7ffdce6b/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.148960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60b555b62194d01a135bfff6e907570, user_id=cd6bd87de8a745eb8e097bbf7ffdce6b, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.151912 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.165605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1358/2752] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:15 2022] PUT /identity/v3/projects/d60b555b62194d01a135bfff6e907570/users/cd6bd87de8a745eb8e097bbf7ffdce6b/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:15.176678 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.176972 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.177249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.203278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] MFA Rules not processed for user `cd6bd87de8a745eb8e097bbf7ffdce6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:15.204480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:15.270530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1395/2753] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:15.281791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.283631 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:15.347918 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aec5c-c571-4b2d-bf12-6f63489483dd admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.351632 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e953a-5fea-4a98-8c74-fbf2be237c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.352282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e953a-5fea-4a98-8c74-fbf2be237c8c service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.352599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e953a-5fea-4a98-8c74-fbf2be237c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.352914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e953a-5fea-4a98-8c74-fbf2be237c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.361896 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e953a-5fea-4a98-8c74-fbf2be237c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.363356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e953a-5fea-4a98-8c74-fbf2be237c8c service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.374151 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1359/2754] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:15.459561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.461170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:15.534295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc836a4d-212f-4048-8220-ca35ca7e5640 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.539254 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.540127 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.540436 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.540808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] PATH_INFO: `/v3/users/a4509851374348e7884055ed11e1a705` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.542201 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4509851374348e7884055ed11e1a705)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.544286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.593039 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4509851374348e7884055ed11e1a705'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:15.600391 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4509851374348e7884055ed11e1a705'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:15.602495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:15.616153 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4509851374348e7884055ed11e1a705'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:15.617689 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4509851374348e7884055ed11e1a705'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:15.619685 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4509851374348e7884055ed11e1a705'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:15.634720 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1396/2755] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:15 2022] DELETE /identity/v3/users/a4509851374348e7884055ed11e1a705 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:15.639400 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.639789 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.640038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.661655 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:15.662771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:15.729086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1360/2756] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:15.739383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:15.740898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:15.797705 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2aa13c-0eb6-48d6-8cb4-2ef075ecbd40 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:15.801366 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429540cd-707c-42e7-ba88-8665d56311a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:15.802009 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429540cd-707c-42e7-ba88-8665d56311a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:15.802295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429540cd-707c-42e7-ba88-8665d56311a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:15.802571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429540cd-707c-42e7-ba88-8665d56311a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:15.811507 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429540cd-707c-42e7-ba88-8665d56311a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:15.812928 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429540cd-707c-42e7-ba88-8665d56311a4 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:15.822810 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1397/2757] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:16.095365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.096843 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:16.109392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3ebbc-c7cd-4eb7-8769-1c747467abdc None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.114161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.115161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.115451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.115734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] PATH_INFO: `/v3/projects/27f02394763340b7be6dcf6152b7d5a3` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.116931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f02394763340b7be6dcf6152b7d5a3)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.119468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.168500 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f02394763340b7be6dcf6152b7d5a3', 'request_id': 'req-690020e3-8382-481f-b8ce-a6e7907bfa31'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:16.175270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f02394763340b7be6dcf6152b7d5a3', 'request_id': 'req-690020e3-8382-481f-b8ce-a6e7907bfa31'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:16.181356 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f02394763340b7be6dcf6152b7d5a3', 'request_id': 'req-690020e3-8382-481f-b8ce-a6e7907bfa31'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:16.193703 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] The token cache is being invalidate because project 27f02394763340b7be6dcf6152b7d5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:16.193987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:16.194605 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-690020e3-8382-481f-b8ce-a6e7907bfa31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:16.196449 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1361/2758] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:16 2022] DELETE /identity/v3/projects/27f02394763340b7be6dcf6152b7d5a3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:16.201879 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.202172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.202457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.220463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:16.221696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:16.282224 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1398/2759] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:16.290185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.290575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.290879 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.297884 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:16.312519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:16.313745 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:16.322191 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1362/2760] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:16.326383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.336013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8d35e-88b1-487f-aaa8-0045c9c7f18d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.340588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.341207 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.341495 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.341769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.342498 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:16:16.342841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.344448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.355835 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1399/2761] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:16:16.360043 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.369584 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dbb23-db3b-4707-aaab-539cf1e0a557 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.374331 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de52b85-51eb-454e-9a97-88c8731e7d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.375007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de52b85-51eb-454e-9a97-88c8731e7d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.375291 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de52b85-51eb-454e-9a97-88c8731e7d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.375570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de52b85-51eb-454e-9a97-88c8731e7d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.376256 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de52b85-51eb-454e-9a97-88c8731e7d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.378145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de52b85-51eb-454e-9a97-88c8731e7d1b admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.403559 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1363/2762] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:16.407641 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.417519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385cf90a-b519-4ab1-9014-329902b0ef51 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.474291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.475003 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.475295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.475592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.476270 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.478839 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.486724 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.487105 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.487511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:16.487791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.500173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1400/2763] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:16.504185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.507829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] MFA Rules not processed for user `96d247f90aa5449dbd54a4ea86458f17`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:16.509837 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:16.514511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f36b-c33a-414c-9780-c391a18c23f2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.519209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.519860 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.520146 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.520426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.521147 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.522753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.535619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1401/2764] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:16.537272 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.548475 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c4d99-4843-4978-b706-1a45afa200e8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.554496 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c4d55-d16d-4d0b-a9b6-86170971830e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.555237 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c4d55-d16d-4d0b-a9b6-86170971830e admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.555526 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c4d55-d16d-4d0b-a9b6-86170971830e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.555828 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c4d55-d16d-4d0b-a9b6-86170971830e admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.557076 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c4d55-d16d-4d0b-a9b6-86170971830e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.558729 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c4d55-d16d-4d0b-a9b6-86170971830e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.568014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1364/2765] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:16.573365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1402/2766] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:16:16.574755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.585874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297664e-be4f-4bc8-8b61-3161b9cd282e None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.592456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.593540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.594005 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.594409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] PATH_INFO: `/v3/projects/bda6f433640643a6acc3c70ec038b1cd/users/a01eabcc36bc469ba94dbc369fba1830/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.611389 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda6f433640643a6acc3c70ec038b1cd, user_id=a01eabcc36bc469ba94dbc369fba1830, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.614052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.631698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1365/2767] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:16 2022] PUT /identity/v3/projects/bda6f433640643a6acc3c70ec038b1cd/users/a01eabcc36bc469ba94dbc369fba1830/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:16.643719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.644084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.644426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.671483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] MFA Rules not processed for user `a01eabcc36bc469ba94dbc369fba1830`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:16.672715 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:16.754471 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1403/2768] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:16.759805 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.840377 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168f4e5-938d-443f-9ba4-551f978b9d4c admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.845579 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.846288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.846701 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.847073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.847860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.849845 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:16.882825 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1366/2769] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:16.888334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:16.900695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92c298-f99c-4140-8f8f-95be04ed3a16 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:16.971264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:16.972078 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:16.972461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:16.972846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:16.974021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:16.976263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.005849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1404/2770] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:16 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:16:17.010227 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.022080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec3d2b-b8fe-4e08-9beb-e0a0ce2bba53 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.027886 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.028471 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.028770 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.029391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.029754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.031570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.045075 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1367/2771] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:17.048969 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.059751 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab6af6-8e4f-45e5-b5d3-9c05aad8bdd2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.065520 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d149be3-fa7a-4ac9-b505-d2bb0c0660f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.066199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d149be3-fa7a-4ac9-b505-d2bb0c0660f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.066484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d149be3-fa7a-4ac9-b505-d2bb0c0660f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.066794 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d149be3-fa7a-4ac9-b505-d2bb0c0660f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.067530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d149be3-fa7a-4ac9-b505-d2bb0c0660f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.069122 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d149be3-fa7a-4ac9-b505-d2bb0c0660f5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.085335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1405/2772] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:17.086554 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.097794 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb55eb5-0e40-47a5-a564-2aecba7710d3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.106115 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.107190 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.107591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.107964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] PATH_INFO: `/v3/projects/727bec91a6b94339be81193de9ec3ce1/users/a29241dad2a04fd2a57c2ff9f7e249af/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.125600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727bec91a6b94339be81193de9ec3ce1, user_id=a29241dad2a04fd2a57c2ff9f7e249af, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.128848 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.148586 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1368/2773] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:17 2022] PUT /identity/v3/projects/727bec91a6b94339be81193de9ec3ce1/users/a29241dad2a04fd2a57c2ff9f7e249af/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:17.221482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.222022 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.222553 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.257753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] MFA Rules not processed for user `a29241dad2a04fd2a57c2ff9f7e249af`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:17.259863 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:17.305534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:17.345888 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1406/2774] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:17.350485 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.357110 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:17.424553 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a906f-af27-4d65-abc0-50c62abcd614 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.429574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.430242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.430529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.430893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.432108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.434018 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.458828 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1369/2775] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:17 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:17.464465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.475176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0d0ed-059f-436f-9ce4-5e3c1920ba9a None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.531300 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.531977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.532263 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.532544 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.533242 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.535206 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.555281 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1407/2776] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:17 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:17.560077 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.570550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98869a2-7ee6-4523-87f7-86ce7b60bed0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.576647 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.577395 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.577680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.577969 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.578797 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.580408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.592127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1370/2777] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:17.596358 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.606353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6667766-c7e4-42e6-a47a-3ce68e17ad13 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.611854 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.612802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.613089 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.613370 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] PATH_INFO: `/v3/projects/949c078fbb2946cba32df28e239ebd0f/users/7f53b52ae9c5473b9551152e27212ebc/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.629674 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=949c078fbb2946cba32df28e239ebd0f, user_id=7f53b52ae9c5473b9551152e27212ebc, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.632812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.646488 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1408/2778] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:17 2022] PUT /identity/v3/projects/949c078fbb2946cba32df28e239ebd0f/users/7f53b52ae9c5473b9551152e27212ebc/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:17.650748 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.660859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3aa96-20f3-45e7-a1fc-f40cddb52556 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.717656 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ccd19-6fa0-4790-a3da-193a390a1598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.718315 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ccd19-6fa0-4790-a3da-193a390a1598 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.718597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ccd19-6fa0-4790-a3da-193a390a1598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.718908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ccd19-6fa0-4790-a3da-193a390a1598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.719627 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ccd19-6fa0-4790-a3da-193a390a1598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.721212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ccd19-6fa0-4790-a3da-193a390a1598 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.730769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1371/2779] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:17.735055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.745736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85654da2-b5de-4101-8883-e57c06001e6a admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.750418 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.751319 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.751681 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.751956 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] PATH_INFO: `/v3/projects/949c078fbb2946cba32df28e239ebd0f/users/7f53b52ae9c5473b9551152e27212ebc/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.754406 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949c078fbb2946cba32df28e239ebd0f, user_id=7f53b52ae9c5473b9551152e27212ebc, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:17.757178 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:17.768088 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1409/2780] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:17 2022] PUT /identity/v3/projects/949c078fbb2946cba32df28e239ebd0f/users/7f53b52ae9c5473b9551152e27212ebc/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:17.778040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.778329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.778737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.804189 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] MFA Rules not processed for user `7f53b52ae9c5473b9551152e27212ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:17.805383 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:17.869525 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:17.877138 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1372/2781] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:17.884752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:17.886447 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:17.942820 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3399313-16d5-4f07-94ba-2650cd8559f0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:17.946372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6551e-389e-4b8c-86de-5aecb2402416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:17.947052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6551e-389e-4b8c-86de-5aecb2402416 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:17.947331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6551e-389e-4b8c-86de-5aecb2402416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:17.947604 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6551e-389e-4b8c-86de-5aecb2402416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:17.981064 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:18.004023 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6551e-389e-4b8c-86de-5aecb2402416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:18.005503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6551e-389e-4b8c-86de-5aecb2402416 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:18.015282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1410/2782] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:18.451503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:18.453011 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:18.516416 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e3bbf-04ed-4441-bb09-77c802648c44 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:18.520965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:18.521831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:18.522114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:18.522390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] PATH_INFO: `/v3/users/cd6bd87de8a745eb8e097bbf7ffdce6b` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:18.523599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd6bd87de8a745eb8e097bbf7ffdce6b)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:18.526059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:18.572209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd6bd87de8a745eb8e097bbf7ffdce6b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:18.579145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd6bd87de8a745eb8e097bbf7ffdce6b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:18.591373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd6bd87de8a745eb8e097bbf7ffdce6b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:18.591998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd6bd87de8a745eb8e097bbf7ffdce6b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:18.592599 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd6bd87de8a745eb8e097bbf7ffdce6b'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:18.604956 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1373/2783] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:18 2022] DELETE /identity/v3/users/cd6bd87de8a745eb8e097bbf7ffdce6b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:18.609420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:18.609711 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:18.609988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:18.626455 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:18.627714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:18.687771 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1411/2784] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:18.697729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:18.699252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:18.756534 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7baa1f-bf88-4fb4-a27b-2583fe0a164e admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:18.761428 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc214183-652b-4862-ac41-d7a42aa2a677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:18.762159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc214183-652b-4862-ac41-d7a42aa2a677 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:18.762446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc214183-652b-4862-ac41-d7a42aa2a677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:18.762755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc214183-652b-4862-ac41-d7a42aa2a677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:18.772492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc214183-652b-4862-ac41-d7a42aa2a677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:18.773986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc214183-652b-4862-ac41-d7a42aa2a677 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:18.784175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1374/2785] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:19.047699 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.049157 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.063938 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d82db-5a69-499e-b80b-df52abd12fcb None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.069119 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.069984 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.070267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.070545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] PATH_INFO: `/v3/projects/d60b555b62194d01a135bfff6e907570` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.071875 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60b555b62194d01a135bfff6e907570)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.073812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.121419 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60b555b62194d01a135bfff6e907570', 'request_id': 'req-709c2cf6-093f-43c0-abad-1c4715e935fb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:19.127503 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60b555b62194d01a135bfff6e907570', 'request_id': 'req-709c2cf6-093f-43c0-abad-1c4715e935fb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:19.133016 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60b555b62194d01a135bfff6e907570', 'request_id': 'req-709c2cf6-093f-43c0-abad-1c4715e935fb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:19.147382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] The token cache is being invalidate because project d60b555b62194d01a135bfff6e907570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:19.147673 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:19.148296 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-709c2cf6-093f-43c0-abad-1c4715e935fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:19.150317 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1412/2786] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:19 2022] DELETE /identity/v3/projects/d60b555b62194d01a135bfff6e907570 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:19.156490 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.156788 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.157069 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.175336 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:19.176528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.193242 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:19.236376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1375/2787] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:19.244968 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.245357 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.245637 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.263912 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:19.265149 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.273245 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1413/2788] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:19.277700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.287402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab7a90-c535-46be-a6dc-f5c7f7f0d944 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.292574 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.293236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.293522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.293799 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.294529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:16:19.294868 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.296890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.308763 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1376/2789] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:16:19.312903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.322426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b48d2-e8b5-4f38-9b81-36d23b15eb0f None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.327647 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.328298 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.328576 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.328848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.329531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.331484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.362571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1414/2790] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:19 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:19.367545 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.372619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.374271 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.378692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5aa92-dd63-4008-b63f-985e9937e16f admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.410501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:19.453050 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3fec-9db8-417f-a2e8-2550124d3851 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.458942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.459514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.459796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.460069 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.461894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.466565 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.467409 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.467699 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.467980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.468695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.470642 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.479725 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.481482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.494549 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1415/2791] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:16:19.503591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1377/2792] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:19 2022] POST /identity/v3/users => generated 282 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:16:19.504780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.520145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.520859 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e220-caa1-46f4-8a1a-112af0a29da7 service placement] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.522000 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.526041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.526737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.527026 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.527305 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.528483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.531084 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.545381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1416/2793] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:19.546785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.558268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd5472-ed36-49df-b017-1bc1c77723f1 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.564310 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.565098 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.565444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.565776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.566590 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.568291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.579571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1417/2794] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:16:19.583429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.586609 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12271fb6-9741-49b4-9e7c-ced992b80dbf admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.590996 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.591651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.591937 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.592225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.593745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2d616-1ca3-4d1a-a87d-8ab333cc2184 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.594014 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.598510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db92ed-76c0-47ba-918f-84fc89931544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.599400 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db92ed-76c0-47ba-918f-84fc89931544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.599684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db92ed-76c0-47ba-918f-84fc89931544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.599967 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db92ed-76c0-47ba-918f-84fc89931544 admin admin] PATH_INFO: `/v3/projects/9085778fe53b40a4b20dee6105833cc4/users/f1a6cb4ec6764f39b5c88b1ef1bf902c/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.615414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db92ed-76c0-47ba-918f-84fc89931544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9085778fe53b40a4b20dee6105833cc4, user_id=f1a6cb4ec6764f39b5c88b1ef1bf902c, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.618074 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db92ed-76c0-47ba-918f-84fc89931544 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.632304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1418/2795] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:19 2022] PUT /identity/v3/projects/9085778fe53b40a4b20dee6105833cc4/users/f1a6cb4ec6764f39b5c88b1ef1bf902c/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:19.652445 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:19.657108 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.658849 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.670220 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1378/2796] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:16:19.698897 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.699286 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.699569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.727179 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] MFA Rules not processed for user `f1a6cb4ec6764f39b5c88b1ef1bf902c`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:19.728425 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.797308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1419/2797] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:19.808506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:19.809943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:19.823543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7af3-2c82-4099-916d-b6d2af47e907 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:19.874096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:19.874782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:19.875062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:19.875335 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:19.884036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:19.886225 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:19.895640 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1379/2798] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:20.201510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:20.211808 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63093b8d-71a0-48c3-9bbf-0c1a5c89adda None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:20.216086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:20.216735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:20.217021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:20.217301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:20.225558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:20.295256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:20.296754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65bba9-3f7a-4a33-ae95-0371a091164a service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:20.300692 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:20.308483 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1420/2799] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:24.171851 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:24.173441 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:24.250061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02bb9-d976-4dfe-85d6-926f59763df8 service cinder] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:24.254815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:24.255696 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:24.255984 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:24.256261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] PATH_INFO: `/v3/users/a01eabcc36bc469ba94dbc369fba1830` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:24.257570 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01eabcc36bc469ba94dbc369fba1830)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:24.260318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:24.313364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01eabcc36bc469ba94dbc369fba1830'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:24.321824 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01eabcc36bc469ba94dbc369fba1830'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:24.333755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01eabcc36bc469ba94dbc369fba1830'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:24.334408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01eabcc36bc469ba94dbc369fba1830'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:24.335052 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01eabcc36bc469ba94dbc369fba1830'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:24.348182 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1380/2800] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:24 2022] DELETE /identity/v3/users/a01eabcc36bc469ba94dbc369fba1830 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:24.353423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:24.353939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:24.354052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:24.371905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:24.373121 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:24.444013 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1421/2801] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:24.456380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:24.457863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:24.485145 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:24.511234 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:24.513278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:24.530665 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d963ad-c30d-4e56-8abd-d319960864db admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:24.534526 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:24.535215 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:24.535503 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:24.535782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:24.547769 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:24.549920 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:24.566919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1381/2802] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:24.577744 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e67386-9fbd-47b0-836f-c390ce5c6b91 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:24.581937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:24.582942 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:24.583480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:24.583732 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:24.585655 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:24.597603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:24.599243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:24.608977 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1422/2803] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 21:16:25.079581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:25.081012 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:25.096277 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991f866-3b5f-44ed-8b9f-0fe14d55d388 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:25.101861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:25.104016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:25.104016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:25.104016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] PATH_INFO: `/v3/projects/bda6f433640643a6acc3c70ec038b1cd` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:25.105622 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda6f433640643a6acc3c70ec038b1cd)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:25.113992 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:25.182099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda6f433640643a6acc3c70ec038b1cd', 'request_id': 'req-fcff180f-f3e5-463f-863b-b22aabd84832'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.189950 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda6f433640643a6acc3c70ec038b1cd', 'request_id': 'req-fcff180f-f3e5-463f-863b-b22aabd84832'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.196735 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda6f433640643a6acc3c70ec038b1cd', 'request_id': 'req-fcff180f-f3e5-463f-863b-b22aabd84832'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.214393 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] The token cache is being invalidate because project bda6f433640643a6acc3c70ec038b1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:25.214801 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.215540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.217339 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1382/2804] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:25 2022] DELETE /identity/v3/projects/bda6f433640643a6acc3c70ec038b1cd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:16:25.221333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:25.235695 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b7c5a-253c-4b9e-b6e6-fc9e2a4bf272 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:25.313519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:25.314804 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:25.315318 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:25.315661 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] PATH_INFO: `/v3/users/a29241dad2a04fd2a57c2ff9f7e249af` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:25.317669 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29241dad2a04fd2a57c2ff9f7e249af)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:25.319727 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:25.374103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:25.377365 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29241dad2a04fd2a57c2ff9f7e249af'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.389758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29241dad2a04fd2a57c2ff9f7e249af'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.404042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29241dad2a04fd2a57c2ff9f7e249af'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.404742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29241dad2a04fd2a57c2ff9f7e249af'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.405512 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29241dad2a04fd2a57c2ff9f7e249af'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.420456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1423/2805] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:25 2022] DELETE /identity/v3/users/a29241dad2a04fd2a57c2ff9f7e249af => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:25.455986 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:25.457238 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:25.569284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff180f-f3e5-463f-863b-b22aabd84832 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:25.643850 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:25.682549 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:25.683265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:25.683550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:25.683829 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:25.685674 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:25.701790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:25.703511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:25.715147 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1383/2806] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:25.788465 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:25.790056 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:25.870297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49469a27-1d06-4e41-8894-3d01a34238b5 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:25.875142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:25.876002 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:25.876283 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:25.876556 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] PATH_INFO: `/v3/projects/727bec91a6b94339be81193de9ec3ce1` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:25.877737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=727bec91a6b94339be81193de9ec3ce1)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:25.879760 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:25.927642 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727bec91a6b94339be81193de9ec3ce1', 'request_id': 'req-038bbbfb-228b-4bfc-b28c-21b683edc355'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.934429 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727bec91a6b94339be81193de9ec3ce1', 'request_id': 'req-038bbbfb-228b-4bfc-b28c-21b683edc355'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.940850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727bec91a6b94339be81193de9ec3ce1', 'request_id': 'req-038bbbfb-228b-4bfc-b28c-21b683edc355'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.954815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] The token cache is being invalidate because project 727bec91a6b94339be81193de9ec3ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:25.955106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.955726 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:25.957269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1424/2807] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:25 2022] DELETE /identity/v3/projects/727bec91a6b94339be81193de9ec3ce1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:25.961775 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:25.972099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6000-c03c-4863-8bd6-4724fe220aac service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.027943 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.028808 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.029097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.029373 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] PATH_INFO: `/v3/users/7f53b52ae9c5473b9551152e27212ebc` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.030540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f53b52ae9c5473b9551152e27212ebc)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.032468 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.077504 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f53b52ae9c5473b9551152e27212ebc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.084097 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f53b52ae9c5473b9551152e27212ebc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.096450 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f53b52ae9c5473b9551152e27212ebc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.097084 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f53b52ae9c5473b9551152e27212ebc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.097704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-4a9e6c9d-5c3d-4ee9-b689-87922f1df55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f53b52ae9c5473b9551152e27212ebc'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.110096 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1384/2808] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:25 2022] DELETE /identity/v3/users/7f53b52ae9c5473b9551152e27212ebc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:26.419477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.422027 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:26.441511 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bbbfb-228b-4bfc-b28c-21b683edc355 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.502080 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:26.519347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.520241 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.520521 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.520799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] PATH_INFO: `/v3/projects/949c078fbb2946cba32df28e239ebd0f` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.523728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=949c078fbb2946cba32df28e239ebd0f)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.526603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.575420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949c078fbb2946cba32df28e239ebd0f', 'request_id': 'req-616d8988-6539-4b77-98d0-c2db2be70554'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.581552 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949c078fbb2946cba32df28e239ebd0f', 'request_id': 'req-616d8988-6539-4b77-98d0-c2db2be70554'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.587042 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949c078fbb2946cba32df28e239ebd0f', 'request_id': 'req-616d8988-6539-4b77-98d0-c2db2be70554'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.598427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] The token cache is being invalidate because project 949c078fbb2946cba32df28e239ebd0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:26.598768 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.599362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-616d8988-6539-4b77-98d0-c2db2be70554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:26.600891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1425/2809] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:26 2022] DELETE /identity/v3/projects/949c078fbb2946cba32df28e239ebd0f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:26.606259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.606761 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.606893 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.625338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:26.626523 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:26.666562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:26.686524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1385/2810] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:26.695218 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.695508 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.695813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.714852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:26.716053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:26.724033 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1426/2811] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:26.728813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.738481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e5a34-1e69-418c-97e6-5074d86223b8 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.743130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.743782 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.744070 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.744349 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] PATH_INFO: `/v3/domains` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.745082 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80749) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:16:26.745390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.747028 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.758436 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1386/2812] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:16:26.762789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.772176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92d70e-eceb-422d-ba1f-51ec8f65acf1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.777360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.778021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.778314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.778589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.779324 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.781238 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.807249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1427/2813] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:16:26 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:26.811381 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.821260 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c8ae-b82e-467a-9fc9-b1da1725668d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.876219 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.877181 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.877469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.877776 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.878497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.880495 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.900945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1387/2814] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:16:26 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:16:26.904929 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.915087 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b4bf8-7a9e-424b-bf01-c8b697f06d65 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.920381 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.920982 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.921267 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.921545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.922273 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.923917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.934083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1428/2815] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:26.937736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.947589 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d584ac-652e-412e-ab83-60005813c3a3 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.952233 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9efd17-20f6-4bcd-bb78-6ca27145d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.952878 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9efd17-20f6-4bcd-bb78-6ca27145d894 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.953165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9efd17-20f6-4bcd-bb78-6ca27145d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.953446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9efd17-20f6-4bcd-bb78-6ca27145d894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:26.954172 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9efd17-20f6-4bcd-bb78-6ca27145d894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:26.955814 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9efd17-20f6-4bcd-bb78-6ca27145d894 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:26.966283 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1388/2816] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:16:26.970106 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:26.980852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99ac14-cd6f-4390-83e0-8cb377ba74b4 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:26.985516 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:26.986393 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:26.986723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:26.987007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] PATH_INFO: `/v3/projects/ac44f6fd5247482cab9e535fbecdf2b8/users/dd25dfa539c94951a59574fd9d2591bb/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:27.001461 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac44f6fd5247482cab9e535fbecdf2b8, user_id=dd25dfa539c94951a59574fd9d2591bb, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:27.004128 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:27.017668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1429/2817] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:16:26 2022] PUT /identity/v3/projects/ac44f6fd5247482cab9e535fbecdf2b8/users/dd25dfa539c94951a59574fd9d2591bb/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:27.022686 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eec5c2-52e3-4a1d-8695-c3292753a81a None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:27.022980 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eec5c2-52e3-4a1d-8695-c3292753a81a None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:27.023286 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eec5c2-52e3-4a1d-8695-c3292753a81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:27.041671 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-66eec5c2-52e3-4a1d-8695-c3292753a81a None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:27.042883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-66eec5c2-52e3-4a1d-8695-c3292753a81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:27.100417 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1389/2818] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:27.110304 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:27.112068 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:27.172490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511dd6-3a16-4514-8a60-00c690e8ce08 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:27.176531 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:27.177174 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:27.177460 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:27.177737 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:27.187736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:27.189320 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:27.199203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1430/2819] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:28.316583 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1fdf8-307f-4daf-887a-084eab46fcf4 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:28.317103 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1fdf8-307f-4daf-887a-084eab46fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:28.317217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1fdf8-307f-4daf-887a-084eab46fcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:28.346491 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-59a1fdf8-307f-4daf-887a-084eab46fcf4 None None] MFA Rules not processed for user `dd25dfa539c94951a59574fd9d2591bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:28.347821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-59a1fdf8-307f-4daf-887a-084eab46fcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:28.392581 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:28.421597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1390/2820] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:28.432910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:28.434480 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:28.458981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:28.499597 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42394e56-7a18-40c1-a054-c817d6d3d89f service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:28.503133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:28.503776 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:28.504058 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:28.504332 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:28.513124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:28.514538 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:28.523170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1431/2821] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:33.017682 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:33.018221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:33.018434 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:33.036372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] MFA Rules not processed for user `dd25dfa539c94951a59574fd9d2591bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:33.037575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:33.045488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1391/2822] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:33.287691 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:33.289135 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:33.300758 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704b42f-729a-4641-a783-ea6f7f5ef7d2 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:33.304450 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:33.305097 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:33.305382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:33.305656 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:33.307464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:33.317774 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:33.319487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7373b98-ae8f-4e19-95c7-aff248e9719e service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:33.330476 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1432/2823] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:34.801970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:34.809765 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:34.940408 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51543881-d29c-4429-a810-31ca9b1424ab None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:34.946107 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:34.947487 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:34.947774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:34.948054 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:34.949861 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:34.962378 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:34.964008 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:34.975324 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1392/2824] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:43.324653 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c304a0-b689-4727-b6db-5b7a7fba47e4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:43.325268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c304a0-b689-4727-b6db-5b7a7fba47e4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:43.325268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c304a0-b689-4727-b6db-5b7a7fba47e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:43.327169 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=23 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:43.374752 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-38c304a0-b689-4727-b6db-5b7a7fba47e4 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:43.375862 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-38c304a0-b689-4727-b6db-5b7a7fba47e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:43.442738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1433/2825] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:43.488117 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:43.489520 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:43.582787 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657b882-1310-4c1d-a9f0-a5540e221527 service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:43.587766 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:43.588414 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:43.588695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:43.588966 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:43.596833 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:43.598403 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:43.606754 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1393/2826] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:16:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:16:44.117911 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7107ba-4239-4ac0-b9fa-cfb212068ac2 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:44.118199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7107ba-4239-4ac0-b9fa-cfb212068ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:44.118497 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7107ba-4239-4ac0-b9fa-cfb212068ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:44.139546 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-8a7107ba-4239-4ac0-b9fa-cfb212068ac2 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:44.140864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8a7107ba-4239-4ac0-b9fa-cfb212068ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:44.149055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1434/2827] 10.222.0.39 () {58 vars in 967 bytes} [Wed Jul 13 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:16:44.161689 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:44.173100 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be172c0-4179-49e5-9024-1d14a20c2857 service swift] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:44.178194 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:44.178860 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:44.179148 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:44.179427 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:44.187481 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:44.189065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:44.197064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1394/2828] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:16:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:16:44.407099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5bb00-2863-4239-b97c-87ff4ee42180 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:44.407558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5bb00-2863-4239-b97c-87ff4ee42180 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:44.407876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5bb00-2863-4239-b97c-87ff4ee42180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:44.425796 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-cad5bb00-2863-4239-b97c-87ff4ee42180 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:44.427134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cad5bb00-2863-4239-b97c-87ff4ee42180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:44.439093 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1435/2829] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:44.453880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:44.464960 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481fb2b-16a4-4a47-b3d0-472470cc33d2 service swift] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:44.470154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:44.470995 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:44.471281 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:44.471556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:44.479880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:44.481447 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:44.489415 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1395/2830] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:16:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:16:44.817134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:44.817430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:44.817716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:44.837501 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:44.839077 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:44.851132 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1436/2831] 10.222.0.39 () {58 vars in 967 bytes} [Wed Jul 13 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:44.866522 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:44.872710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=100 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:44.883062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b4c38-2165-44d4-b3f2-74b070d998e0 service swift] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:44.890831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03321e-3adc-4533-89b7-74b0c5f57dee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:44.892073 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03321e-3adc-4533-89b7-74b0c5f57dee service swift] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:44.892405 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03321e-3adc-4533-89b7-74b0c5f57dee service swift] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:44.893002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03321e-3adc-4533-89b7-74b0c5f57dee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:44.902486 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03321e-3adc-4533-89b7-74b0c5f57dee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:44.904050 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03321e-3adc-4533-89b7-74b0c5f57dee service swift] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:44.912376 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1396/2832] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:16:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:16:46.984714 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:46.987142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:47.000858 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391caf8-910c-4a8d-8e45-b53b6ec9be3d None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:47.005386 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:47.006185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:47.006473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:47.006789 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] PATH_INFO: `/v3/users/96d247f90aa5449dbd54a4ea86458f17` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:47.007959 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d247f90aa5449dbd54a4ea86458f17)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:47.010116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:47.055675 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d247f90aa5449dbd54a4ea86458f17'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.062551 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d247f90aa5449dbd54a4ea86458f17'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.073921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d247f90aa5449dbd54a4ea86458f17'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.074545 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d247f90aa5449dbd54a4ea86458f17'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.075172 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d247f90aa5449dbd54a4ea86458f17'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.086920 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1437/2833] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:16:46 2022] DELETE /identity/v3/users/96d247f90aa5449dbd54a4ea86458f17 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:47.091862 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:47.092348 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:47.092448 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:47.110328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:47.111577 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:47.170895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1397/2834] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:47.180338 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:47.181785 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:47.213487 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:16:47.247980 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945930a-c232-42fb-99c3-91f8f694c4b2 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:47.251658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544cd70-8845-4b68-8f65-168d12bbe643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:47.252330 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cd70-8845-4b68-8f65-168d12bbe643 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:47.252616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cd70-8845-4b68-8f65-168d12bbe643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:47.252891 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544cd70-8845-4b68-8f65-168d12bbe643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:47.263131 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cd70-8845-4b68-8f65-168d12bbe643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:47.264505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544cd70-8845-4b68-8f65-168d12bbe643 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:47.276562 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1438/2835] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:16:47.445278 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:47.446813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:47.459675 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d650a90-638f-4593-a58e-fce08d929797 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:47.465540 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:47.466698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:47.466895 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:47.467178 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] PATH_INFO: `/v3/projects/908836d993404006970ed39a9613b600` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:47.468372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] RBAC: Authorizing `identity:delete_project(project_id=908836d993404006970ed39a9613b600)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:47.470330 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:47.517737 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908836d993404006970ed39a9613b600', 'request_id': 'req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.523831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908836d993404006970ed39a9613b600', 'request_id': 'req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.531528 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908836d993404006970ed39a9613b600', 'request_id': 'req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.546067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] The token cache is being invalidate because project 908836d993404006970ed39a9613b600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:16:47.546402 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.547118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:16:47.554263 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1398/2836] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:16:47 2022] DELETE /identity/v3/projects/908836d993404006970ed39a9613b600 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:16:52.317252 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:52.317709 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:52.317805 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:52.335073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] MFA Rules not processed for user `dd25dfa539c94951a59574fd9d2591bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:16:52.336257 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:52.396747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1439/2837] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:16:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:16:52.624252 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:16:52.625713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:52.681964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c6ea2-d480-4bb0-aee1-fe7ffe2a465d admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:16:52.685710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:16:52.686391 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:16:52.686710 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:16:52.686991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:16:52.688771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:16:52.698282 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:16:52.700235 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e24e13-72ce-4dad-a112-fd15aafd917e service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:16:52.708154 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1399/2838] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:16:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:02.375368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:02.376763 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:02.439342 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b681315-bcc7-4231-a0a7-b48e3a5c2611 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:02.443762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:02.444612 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:02.444894 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:02.445170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] PATH_INFO: `/v3/users/f1a6cb4ec6764f39b5c88b1ef1bf902c` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:02.446326 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a6cb4ec6764f39b5c88b1ef1bf902c)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:02.448481 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:02.493668 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a6cb4ec6764f39b5c88b1ef1bf902c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:02.501256 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a6cb4ec6764f39b5c88b1ef1bf902c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:02.513250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a6cb4ec6764f39b5c88b1ef1bf902c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:02.513882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a6cb4ec6764f39b5c88b1ef1bf902c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:02.514484 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a6cb4ec6764f39b5c88b1ef1bf902c'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:02.516477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:02.528846 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1440/2839] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:17:02 2022] DELETE /identity/v3/users/f1a6cb4ec6764f39b5c88b1ef1bf902c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:02.533317 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:02.533790 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:02.533906 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:02.551901 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:02.553118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:02.562209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:02.613294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1400/2840] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:02.622812 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:02.624196 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:02.680055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786c27a-7fc0-4b77-b4d4-119181aeaf86 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:02.683735 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d108a-d1cc-4ba8-a137-215cc2e1dd52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:02.684376 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d108a-d1cc-4ba8-a137-215cc2e1dd52 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:02.684684 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d108a-d1cc-4ba8-a137-215cc2e1dd52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:02.684964 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d108a-d1cc-4ba8-a137-215cc2e1dd52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:02.694158 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d108a-d1cc-4ba8-a137-215cc2e1dd52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:02.695628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d108a-d1cc-4ba8-a137-215cc2e1dd52 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:02.705628 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1441/2841] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:03.011267 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.012767 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:03.028949 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da8311-d6c7-45a8-99a1-84d4de415725 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.034598 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.035575 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.035931 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.036270 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] PATH_INFO: `/v3/projects/9085778fe53b40a4b20dee6105833cc4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.037521 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9085778fe53b40a4b20dee6105833cc4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.039648 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.098558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9085778fe53b40a4b20dee6105833cc4', 'request_id': 'req-693bbf2c-deeb-4c89-91c0-48345e36f9a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:03.105137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9085778fe53b40a4b20dee6105833cc4', 'request_id': 'req-693bbf2c-deeb-4c89-91c0-48345e36f9a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:03.111718 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9085778fe53b40a4b20dee6105833cc4', 'request_id': 'req-693bbf2c-deeb-4c89-91c0-48345e36f9a1'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:03.126497 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] The token cache is being invalidate because project 9085778fe53b40a4b20dee6105833cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:17:03.126998 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:03.127854 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-693bbf2c-deeb-4c89-91c0-48345e36f9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:03.130120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1401/2842] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:17:03 2022] DELETE /identity/v3/projects/9085778fe53b40a4b20dee6105833cc4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:03.135368 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.136593 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.137086 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.161020 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:03.162220 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:03.234382 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1442/2843] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 21:17:03.242165 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.242700 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.243080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.263457 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:03.265367 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:03.278063 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1402/2844] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:03.281886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.295148 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18491445-37e5-4419-8409-7af2cb7f0afd None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.300103 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.300801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.301115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.301399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.302264 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:17:03.302583 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.304282 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.317430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1443/2845] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:17:03.321764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.332126 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54c7a-b493-4efd-b3ba-8736295dfdbd None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.337061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.337723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.338010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.338289 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.339035 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.340987 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.382668 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1403/2846] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:17:03 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:03.383937 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.396032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97fcc03-fe73-43b1-bbf3-2541649d693f admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.458924 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.460108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.460108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.460108 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.460801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.462793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.483795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1444/2847] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:17:03 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:03.488047 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.499224 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90598801-9a37-456a-b82e-6fbf4e1807e0 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.504071 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.504891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.505081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.505398 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.506357 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.508022 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.520736 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1404/2848] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:03.524983 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.537117 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093088c6-089e-4a43-a9d8-5e8bdc14377e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.541728 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68609d00-0e47-403b-bbdb-0cfe52abd7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.542379 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68609d00-0e47-403b-bbdb-0cfe52abd7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.542690 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68609d00-0e47-403b-bbdb-0cfe52abd7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.542995 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68609d00-0e47-403b-bbdb-0cfe52abd7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.543719 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68609d00-0e47-403b-bbdb-0cfe52abd7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.545291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68609d00-0e47-403b-bbdb-0cfe52abd7c6 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.550430 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:03.559202 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1445/2849] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 21:17:03.561638 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.571815 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fda8d7-1d06-45d5-a524-938ea58802ae admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.577099 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.577978 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.578265 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.578543 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] PATH_INFO: `/v3/projects/3ed08895dc72486783da247550696d4c/users/72ad69b2fab34a6180d7353c1aa730d3/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.594353 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed08895dc72486783da247550696d4c, user_id=72ad69b2fab34a6180d7353c1aa730d3, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.597033 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.610290 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1405/2850] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:17:03 2022] PUT /identity/v3/projects/3ed08895dc72486783da247550696d4c/users/72ad69b2fab34a6180d7353c1aa730d3/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:03.615279 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6f77e-736c-4706-9543-d91d14d3201c None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.615571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6f77e-736c-4706-9543-d91d14d3201c None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.615850 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6f77e-736c-4706-9543-d91d14d3201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.632522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-72e6f77e-736c-4706-9543-d91d14d3201c None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:03.633716 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-72e6f77e-736c-4706-9543-d91d14d3201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:03.691840 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1446/2851] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:03.703040 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:03.704747 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:03.744159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:03.763512 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609e154-08ec-403d-a0e5-dbf1de59ca93 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:03.767118 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:03.767755 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:03.768034 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:03.768305 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:03.777114 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:03.779109 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:03.788437 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1406/2852] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:08.072953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.073375 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.075144 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.105756 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] MFA Rules not processed for user `72ad69b2fab34a6180d7353c1aa730d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:08.107529 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:08.191456 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1447/2853] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:08.195255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:08.205596 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66129c9-02e9-4666-a50e-d8ffcc1ea158 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:08.210970 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:08.211618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.212131 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.212707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.213713 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:08.216051 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:08.245261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1407/2854] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:17:08 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:08.250191 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:08.262457 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329824f6-6c65-4556-bbde-4b645a4b9ae3 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:08.351295 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:08.352021 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.352307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.352588 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.353745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:08.355609 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:08.382352 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1448/2855] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:17:08 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:08.383695 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:08.395132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff7796-313f-4eb0-9178-d880318964e8 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:08.399890 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:08.400564 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.400842 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.401132 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.401863 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:08.404167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:08.415061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1408/2856] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:08.419510 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:08.432783 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168176a-81ee-402e-96ea-75c0c8a5baca admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:08.438874 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:08.439754 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.440036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.440307 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] PATH_INFO: `/v3/projects/991f25641b8d4792923785ee02106fe4/users/74a3720008ef4e9392852651506fab0b/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.455670 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991f25641b8d4792923785ee02106fe4, user_id=74a3720008ef4e9392852651506fab0b, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:08.459004 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:08.462134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:08.475095 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1449/2857] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:17:08 2022] PUT /identity/v3/projects/991f25641b8d4792923785ee02106fe4/users/74a3720008ef4e9392852651506fab0b/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:08.479067 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:08.488964 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ae7f8-a22c-4a9c-8b74-c35bb7de2fae admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:08.544891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff14b7d-2232-46f9-a60f-c81660eb5dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:08.545562 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff14b7d-2232-46f9-a60f-c81660eb5dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.545859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff14b7d-2232-46f9-a60f-c81660eb5dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.546130 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff14b7d-2232-46f9-a60f-c81660eb5dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.546910 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff14b7d-2232-46f9-a60f-c81660eb5dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:08.548558 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff14b7d-2232-46f9-a60f-c81660eb5dc8 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:08.558223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1409/2858] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:08.562845 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:08.573175 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25003971-edad-4ea4-8875-bffe6703af60 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:08.578486 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:08.579415 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] REQUEST_METHOD: `PUT` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:08.579696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:08.579970 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] PATH_INFO: `/v3/projects/991f25641b8d4792923785ee02106fe4/users/74a3720008ef4e9392852651506fab0b/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:08.581974 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] RBAC: Authorizing `identity:create_grant(project_id=991f25641b8d4792923785ee02106fe4, user_id=74a3720008ef4e9392852651506fab0b, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:08.584644 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:08.595522 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1450/2859] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:17:08 2022] PUT /identity/v3/projects/991f25641b8d4792923785ee02106fe4/users/74a3720008ef4e9392852651506fab0b/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:12.869772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31050-dee7-4fb9-adef-192d2d307246 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:12.870350 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31050-dee7-4fb9-adef-192d2d307246 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:12.870557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31050-dee7-4fb9-adef-192d2d307246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:12.918171 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-a0a31050-dee7-4fb9-adef-192d2d307246 None None] MFA Rules not processed for user `74a3720008ef4e9392852651506fab0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:12.919704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-a0a31050-dee7-4fb9-adef-192d2d307246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:13.025478 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1410/2860] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:13.040262 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:13.042053 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:13.164427 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c77a7-fa42-4850-9e2a-86beff9f899e admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:13.171334 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:13.172173 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:13.172571 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:13.172988 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:13.239116 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:13.240886 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:13.255115 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1451/2861] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:15.175880 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a660-aaf9-42be-8982-44b192807c83 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:15.176430 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a660-aaf9-42be-8982-44b192807c83 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:15.176872 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30a660-aaf9-42be-8982-44b192807c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:15.199813 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-bb30a660-aaf9-42be-8982-44b192807c83 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:15.201442 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-bb30a660-aaf9-42be-8982-44b192807c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:15.226779 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:15.270102 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1411/2862] 10.222.0.39 () {58 vars in 1009 bytes} [Wed Jul 13 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:15.281083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:15.282477 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:15.356625 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fc3d-5ff4-4509-a3d6-792042665607 service nova] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:15.362167 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:15.362939 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:15.363803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:15.364203 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:15.374337 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:15.376044 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:15.387227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1452/2863] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:17:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:17:25.259205 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=13 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:25.383154 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:36.078791 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:36.079264 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:36.079365 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:36.097462 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] MFA Rules not processed for user `4535c7d4c7a74095b9ccc6777ffee724`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:36.098692 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:36.105940 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1412/2864] 10.222.0.39 () {58 vars in 967 bytes} [Wed Jul 13 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:36.118293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:36.129426 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95692a-6644-4228-b4c3-b57c424c312d service swift] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:36.134534 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '292fed9f5de945c883cc1ffeb91da4cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:36.135468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:36.135747 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:36.136019 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:36.144221 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:36.145847 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:36.154453 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1453/2865] 10.222.0.39 () {60 vars in 1384 bytes} [Wed Jul 13 21:17:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:17:37.966175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:37.967641 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:38.028793 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.030424 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:38.046062 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4ddc-4a00-4463-a9eb-1482e82e7167 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.051432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.052166 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.052529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.052883 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.055039 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:38.067721 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.069244 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.080049 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1413/2866] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:38.103047 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1261a5e-a33e-41fb-aec1-7379fef1c1aa service swift] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.108327 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.109142 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.109423 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.109797 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] PATH_INFO: `/v3/users/dd25dfa539c94951a59574fd9d2591bb` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.110917 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd25dfa539c94951a59574fd9d2591bb)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.112841 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.167541 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd25dfa539c94951a59574fd9d2591bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.175356 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd25dfa539c94951a59574fd9d2591bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.188333 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd25dfa539c94951a59574fd9d2591bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.188978 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd25dfa539c94951a59574fd9d2591bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.189587 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-94572f18-2e7e-498c-bcb9-c4dcd6867059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd25dfa539c94951a59574fd9d2591bb'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.201564 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1454/2867] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:17:38 2022] DELETE /identity/v3/users/dd25dfa539c94951a59574fd9d2591bb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:17:38.370397 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.381467 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c6ff2-7e8f-4fc6-84c8-4d76367a3b81 service placement] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.440432 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.441288 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.441580 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.441859 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] PATH_INFO: `/v3/projects/ac44f6fd5247482cab9e535fbecdf2b8` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.443065 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac44f6fd5247482cab9e535fbecdf2b8)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.444967 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.488591 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac44f6fd5247482cab9e535fbecdf2b8', 'request_id': 'req-bfd2d180-d11f-43e5-91b2-74929e0c16db'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.494460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac44f6fd5247482cab9e535fbecdf2b8', 'request_id': 'req-bfd2d180-d11f-43e5-91b2-74929e0c16db'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.500472 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac44f6fd5247482cab9e535fbecdf2b8', 'request_id': 'req-bfd2d180-d11f-43e5-91b2-74929e0c16db'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.512328 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] The token cache is being invalidate because project ac44f6fd5247482cab9e535fbecdf2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80749) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:17:38.512651 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.513251 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-bfd2d180-d11f-43e5-91b2-74929e0c16db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:17:38.514832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1414/2868] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:17:38 2022] DELETE /identity/v3/projects/ac44f6fd5247482cab9e535fbecdf2b8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:38.519775 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.520066 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.520409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.537991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:38.539214 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:38.597227 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1455/2869] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:38.604928 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.605223 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.605506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.625623 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:38.626798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:38.634555 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1415/2870] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:38.639301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.648903 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0cec-93cf-4e1b-a170-cc30361983f4 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.653473 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.654124 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.654411 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.654721 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] PATH_INFO: `/v3/domains` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.655448 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80748) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:17:38.655759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.657374 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.658849 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:38.675801 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1456/2871] 10.222.0.39 () {58 vars in 1152 bytes} [Wed Jul 13 21:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:17:38.676853 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.686392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851fc3d-1c67-413c-95e7-65684e83bbe0 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.691007 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.691652 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.691935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.692213 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.692908 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.694836 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.697977 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:38.723539 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1416/2872] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:17:38 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:38.727832 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.738268 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf994bc-1277-4ef4-8fbc-398c2afa3914 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.791616 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.792277 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.792558 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.792831 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.793514 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.795991 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.814362 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1457/2873] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:17:38 2022] POST /identity/v3/users => generated 287 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:17:38.818469 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.828550 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef346d3-db4c-410b-b867-ab9fc045eb99 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.833127 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.833780 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.834064 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.834340 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.835104 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.836685 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.847390 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1417/2874] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:38.851025 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.861246 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e701434-f59c-41aa-8093-1e3a2da7bab0 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.865923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa39d7-35e2-499a-96c5-c92f9b9eefc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.866698 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa39d7-35e2-499a-96c5-c92f9b9eefc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.866935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa39d7-35e2-499a-96c5-c92f9b9eefc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.867209 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa39d7-35e2-499a-96c5-c92f9b9eefc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.867932 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa39d7-35e2-499a-96c5-c92f9b9eefc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.869505 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa39d7-35e2-499a-96c5-c92f9b9eefc4 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.879341 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1458/2875] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:38.883193 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:38.894338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aca172-0a87-4e30-af1f-46b008867266 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:38.898905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:38.899758 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.900036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.900307 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] PATH_INFO: `/v3/projects/59ed37e30e7b4a7188823efe184af5a1/users/262475732e324ecbbf1bb54c33fcc425/roles/8182238074534b76a65d54c2edbeffad` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.915832 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ed37e30e7b4a7188823efe184af5a1, user_id=262475732e324ecbbf1bb54c33fcc425, role_id=8182238074534b76a65d54c2edbeffad)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:38.918517 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:38.930492 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1418/2876] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:17:38 2022] PUT /identity/v3/projects/59ed37e30e7b4a7188823efe184af5a1/users/262475732e324ecbbf1bb54c33fcc425/roles/8182238074534b76a65d54c2edbeffad => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:38.935718 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5635fa-1dbc-46d8-8c16-48e17962e7b0 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:38.936040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5635fa-1dbc-46d8-8c16-48e17962e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:38.936322 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5635fa-1dbc-46d8-8c16-48e17962e7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:38.953803 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-aa5635fa-1dbc-46d8-8c16-48e17962e7b0 None None] MFA Rules not processed for user `a762106a78c649808cb7ea162b5997db`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:38.955029 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-aa5635fa-1dbc-46d8-8c16-48e17962e7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:39.011335 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1459/2877] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:39.021413 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:39.023230 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:39.079678 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6d8c-730c-483f-ae9c-7c061e5a4973 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:39.083259 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009dfaf-4189-4b67-bd26-714ff50c275b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:39.083898 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009dfaf-4189-4b67-bd26-714ff50c275b service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:39.084179 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009dfaf-4189-4b67-bd26-714ff50c275b service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:39.084451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009dfaf-4189-4b67-bd26-714ff50c275b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:39.094098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009dfaf-4189-4b67-bd26-714ff50c275b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:39.095759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009dfaf-4189-4b67-bd26-714ff50c275b service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:39.104919 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1419/2878] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:40.334913 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:40.336388 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:40.336986 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:40.367028 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] MFA Rules not processed for user `72ad69b2fab34a6180d7353c1aa730d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:40.375151 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:40.444383 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1460/2879] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:43.761384 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:43.761945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:43.762318 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:43.799557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] MFA Rules not processed for user `262475732e324ecbbf1bb54c33fcc425`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:43.801137 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:43.891241 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1420/2880] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:43.892599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:43.905482 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81b7d-4812-4fb4-a62d-4deea1937fba None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:43.910422 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:43.911346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:43.911802 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:43.912085 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:43.912821 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:43.914795 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:43.930217 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:43.950708 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1461/2881] 10.222.0.39 () {60 vars in 1152 bytes} [Wed Jul 13 21:17:43 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:43.951921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:43.963749 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e6387-e8fc-47c2-b4e0-a40297121896 None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:44.056217 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:44.079086 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:44.080200 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:44.080653 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:44.081133 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] PATH_INFO: `/v3/users` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:44.082010 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:44.084209 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:44.112299 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1421/2882] 10.222.0.39 () {60 vars in 1143 bytes} [Wed Jul 13 21:17:43 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:17:44.116490 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:44.128372 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bab397-bd11-4d99-a403-b541ab211345 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:44.133953 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:44.134759 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:44.135243 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:44.135527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:44.136293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:44.138444 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:44.150651 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1462/2883] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:44.155561 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:44.166798 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e602d4-5091-45a2-99ef-ed3f47817fef admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:44.172359 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:44.173183 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:44.173519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:44.173915 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] PATH_INFO: `/v3/projects/048616df98d8453bb82bc506c8fbf465/users/f06e2f1b375045ec9d0254cb70d1d5c4/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:44.192159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048616df98d8453bb82bc506c8fbf465, user_id=f06e2f1b375045ec9d0254cb70d1d5c4, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:44.194913 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:44.211461 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1422/2884] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:17:44 2022] PUT /identity/v3/projects/048616df98d8453bb82bc506c8fbf465/users/f06e2f1b375045ec9d0254cb70d1d5c4/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:17:44.215420 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:44.226519 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c0590-a1ab-4ebb-857e-b7217b24e0ab admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:44.295837 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e421e195-4c17-4ff3-ac2e-3d8ffad9e4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:44.296543 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421e195-4c17-4ff3-ac2e-3d8ffad9e4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:44.296815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421e195-4c17-4ff3-ac2e-3d8ffad9e4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:44.297141 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e421e195-4c17-4ff3-ac2e-3d8ffad9e4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:44.297880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421e195-4c17-4ff3-ac2e-3d8ffad9e4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:44.299819 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e421e195-4c17-4ff3-ac2e-3d8ffad9e4e7 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:44.311120 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1463/2885] 10.222.0.39 () {58 vars in 1121 bytes} [Wed Jul 13 21:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 21:17:44.317961 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:44.332488 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8514-29b7-4183-849e-a52003319cd9 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:44.337905 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:44.338873 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:44.339158 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:44.339431 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] PATH_INFO: `/v3/projects/048616df98d8453bb82bc506c8fbf465/users/f06e2f1b375045ec9d0254cb70d1d5c4/roles/685cce0e789945d8ad7e2d0fab42e194` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:44.342199 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048616df98d8453bb82bc506c8fbf465, user_id=f06e2f1b375045ec9d0254cb70d1d5c4, role_id=685cce0e789945d8ad7e2d0fab42e194)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:44.345280 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:44.367212 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1423/2886] 10.222.0.39 () {60 vars in 1482 bytes} [Wed Jul 13 21:17:44 2022] PUT /identity/v3/projects/048616df98d8453bb82bc506c8fbf465/users/f06e2f1b375045ec9d0254cb70d1d5c4/roles/685cce0e789945d8ad7e2d0fab42e194 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:17:49.288259 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396c707-6807-452b-ab35-70aa9a94fa79 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:49.288667 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396c707-6807-452b-ab35-70aa9a94fa79 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:49.288864 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396c707-6807-452b-ab35-70aa9a94fa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:49.323957 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-9396c707-6807-452b-ab35-70aa9a94fa79 None None] MFA Rules not processed for user `f06e2f1b375045ec9d0254cb70d1d5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:49.326882 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-9396c707-6807-452b-ab35-70aa9a94fa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:49.416935 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1464/2887] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:49.430338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:49.432161 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:49.513835 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ca09c-7cb6-4a9b-bb04-d49ca9cf33a9 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:49.519332 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:49.519979 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:49.520261 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:49.520532 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:49.576942 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:49.578460 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:49.588354 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1424/2888] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:17:50.682603 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:50.683139 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:50.683405 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:50.701185 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] MFA Rules not processed for user `262475732e324ecbbf1bb54c33fcc425`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:17:50.702399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:50.709399 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1465/2889] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:17:59.051869 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:17:59.053392 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:59.087121 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:17:59.110965 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c88834-41a6-4e10-973e-3ea5591dedaf service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:17:59.114511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:17:59.115175 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:17:59.115456 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:17:59.115732 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:17:59.117476 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:17:59.180891 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:17:59.182306 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:17:59.192125 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1425/2890] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:17:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:18:00.695815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=73 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:18:09.187921 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=36 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:18:27.150297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:18:27.160589 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf015bc-81e2-4804-8d7b-a75a63887a64 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:18:27.164170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:18:27.164813 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:18:27.165099 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:18:27.165378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:18:27.174848 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:18:27.176269 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:18:27.185736 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1466/2891] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:18:37.181779 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:18:49.999704 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:18:50.001578 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:18:50.069484 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50872-192a-4822-b015-d843a4ebd952 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:18:50.074506 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:18:50.075207 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:18:50.075494 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:18:50.075772 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:18:50.077544 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:18:50.089458 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:18:50.091078 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:18:50.101038 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1426/2892] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:18:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:19:00.096764 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=32 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:20:14.920981 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:20:14.929798 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63612f-ad27-4876-bfb7-b8d7b8a200cc service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:20:14.933530 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:20:14.934165 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:20:14.934452 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:20:14.934773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:20:14.936574 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:20:14.950321 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:20:14.951792 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03bd1-add9-408f-938f-d22705c05415 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:20:14.963433 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1467/2893] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:20:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:20:16.137897 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:20:16.148834 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ccab-368e-4719-9daf-c6cc2a6102ee service placement] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:20:16.152593 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:20:16.153239 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:20:16.153524 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:20:16.153803 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:20:16.155607 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:20:16.170129 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:20:16.171617 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:20:16.182426 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1427/2894] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:20:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:20:18.459954 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] REQUEST_METHOD: `POST` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:20:18.460464 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:20:18.460573 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:20:18.478308 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.auth.core [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] MFA Rules not processed for user `72ad69b2fab34a6180d7353c1aa730d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80748) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:20:18.479619 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:20:18.538250 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1468/2895] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:20:26.177971 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:20:28.530361 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=31 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:21:03.940247 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:21:03.949531 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd049f-048b-4ae1-876b-a5a579e64a82 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:21:03.953945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:21:03.954774 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:21:03.955059 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:21:03.955329 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:21:03.957962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:21:03.969044 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:21:03.970475 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:21:03.979380 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1428/2896] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:21:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:21:05.101513 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:21:05.111434 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387ac5-109f-4320-aae6-2761891a38b1 None None] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:21:05.115007 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:21:05.115640 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:21:05.115921 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:21:05.116199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:21:05.118722 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:21:05.132693 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:21:05.134040 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:21:05.145542 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1469/2897] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:21:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:21:13.975061 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:21:15.141274 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:21:55.684511 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:21:55.693185 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e443f-8a7b-4ae3-9134-9d8523760228 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:21:55.697372 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:21:55.697516 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:21:55.697751 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:21:55.698031 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:21:55.700002 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:21:55.709618 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:21:55.711597 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5408d-a842-4b66-90ac-e5ba8a8f573a service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:21:55.719621 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1429/2898] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:21:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:22:05.715935 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:22:24.379398 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:22:24.380815 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:24.439306 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9a5ad-7af8-4d8a-baab-ff341605838b service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:22:24.442834 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a58cb03e518413ab12cc84d793c4736', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:22:24.443478 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:24.443762 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:24.444038 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:24.445961 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:24.457898 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:22:24.459346 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:22:24.468694 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1470/2899] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:22:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:22:34.464683 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=26 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:22:35.414707 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] REQUEST_METHOD: `POST` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:35.415198 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:35.415311 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:35.432698 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.auth.core [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] MFA Rules not processed for user `72ad69b2fab34a6180d7353c1aa730d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80749) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:22:35.433874 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:35.440573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1430/2900] 10.222.0.39 () {58 vars in 957 bytes} [Wed Jul 13 21:22:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 21:22:35.753638 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:22:35.763092 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be4f18-9d71-4a3d-b4c9-5509fcc4e614 service cinder] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:22:35.766723 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:22:35.767353 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:35.767635 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:35.767905 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:35.769658 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:35.778876 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:22:35.780278 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:22:35.788136 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1471/2901] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:22:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:22:36.218687 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:22:36.231081 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bbccc-0167-48a1-a52e-13eb4b24401f None None] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:22:36.236080 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:22:36.236734 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:36.237016 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:36.237295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:36.239236 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:36.251446 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:22:36.253058 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:22:36.264309 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1431/2902] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:22:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:22:38.864738 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:22:38.866248 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:38.885170 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b410-65d4-411e-b78b-7b7f29114eeb service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:22:38.891360 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:22:38.892043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:38.892396 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:38.892742 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:38.894592 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:38.909083 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:22:38.911055 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:22:38.933226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1472/2903] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:22:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:22:46.259136 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=9 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:22:48.918199 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=12 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:22:51.245786 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:22:51.259723 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c7be8-3266-4fc4-9c3f-5697905afda5 service placement] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:22:51.266294 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2048d28ca3b42d8b8d82f4754e44433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:22:51.266981 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:51.267262 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:51.267537 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:51.269364 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:22:51.284991 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:22:51.286573 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:22:51.298254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1432/2904] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:22:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:22:52.892923 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:22:52.902291 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c688f35-c872-4698-8bc4-5143ed12502e service nova] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:22:52.905880 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:22:52.906527 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:22:52.906852 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:22:52.907133 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:22:52.915350 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:22:52.916769 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:22:52.925459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1473/2905] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:22:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:23:00.119784 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:00.129733 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cccf5-1a71-4c59-b12d-28e5a3664ad6 service nova] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:00.133529 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:00.134159 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:00.134444 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:00.134759 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:00.195255 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:00.196714 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:00.206268 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1433/2906] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:23:02.921524 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=6 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:23:02.969249 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:02.981255 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e6a81-aec2-4c3c-b595-348e162bbc29 service neutron] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:02.986293 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:02.987041 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:02.987403 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:02.987753 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:02.989617 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:23:03.002129 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:03.003773 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:03.016176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1474/2907] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:23:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:23:06.146989 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:06.157421 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14560479-4374-4169-86e1-657d7f61ccf3 service neutron] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:06.162030 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:06.162934 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:06.163221 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:06.163508 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] PATH_INFO: `/v3/users/262475732e324ecbbf1bb54c33fcc425` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:06.164711 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=262475732e324ecbbf1bb54c33fcc425)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:06.166680 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:06.218399 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262475732e324ecbbf1bb54c33fcc425'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.228042 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262475732e324ecbbf1bb54c33fcc425'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.241254 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262475732e324ecbbf1bb54c33fcc425'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.241988 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262475732e324ecbbf1bb54c33fcc425'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.242595 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262475732e324ecbbf1bb54c33fcc425'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.255781 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1434/2908] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:23:06 2022] DELETE /identity/v3/users/262475732e324ecbbf1bb54c33fcc425 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:23:06.440297 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:06.455193 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cebf60-9842-4dcf-ad94-af0656adb8f5 service placement] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:06.511539 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:06.513052 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:06.513331 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:06.513599 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] PATH_INFO: `/v3/projects/59ed37e30e7b4a7188823efe184af5a1` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:06.514799 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ed37e30e7b4a7188823efe184af5a1)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:06.516696 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:06.562073 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ed37e30e7b4a7188823efe184af5a1', 'request_id': 'req-90e9400b-142b-43a8-81ea-6b9ded823946'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.568005 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ed37e30e7b4a7188823efe184af5a1', 'request_id': 'req-90e9400b-142b-43a8-81ea-6b9ded823946'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.573786 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ed37e30e7b4a7188823efe184af5a1', 'request_id': 'req-90e9400b-142b-43a8-81ea-6b9ded823946'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.585660 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] The token cache is being invalidate because project 59ed37e30e7b4a7188823efe184af5a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:23:06.585948 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.586569 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.588107 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1475/2909] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:23:06 2022] DELETE /identity/v3/projects/59ed37e30e7b4a7188823efe184af5a1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:23:06.592168 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:06.593557 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:23:06.606821 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb81e85-0c0b-4d3c-8433-47847df378e2 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:06.653120 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:23:06.663945 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:06.664818 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:06.665098 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:06.665369 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] PATH_INFO: `/v3/users/f06e2f1b375045ec9d0254cb70d1d5c4` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:06.666556 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06e2f1b375045ec9d0254cb70d1d5c4)` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:06.669036 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:06.712827 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06e2f1b375045ec9d0254cb70d1d5c4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.719333 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06e2f1b375045ec9d0254cb70d1d5c4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.731812 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06e2f1b375045ec9d0254cb70d1d5c4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.732453 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06e2f1b375045ec9d0254cb70d1d5c4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.733085 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.notifications [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06e2f1b375045ec9d0254cb70d1d5c4'} {{(pid=80749) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:06.744946 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1435/2910] 10.222.0.39 () {58 vars in 1223 bytes} [Wed Jul 13 21:23:06 2022] DELETE /identity/v3/users/f06e2f1b375045ec9d0254cb70d1d5c4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:23:06.922032 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:06.923780 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:23:06.937993 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9400b-142b-43a8-81ea-6b9ded823946 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:06.995468 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a762106a78c649808cb7ea162b5997db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c113e9e1592248a9b12fd4d2b385aa5a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:06.996409 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:06.996806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:06.997091 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] PATH_INFO: `/v3/projects/048616df98d8453bb82bc506c8fbf465` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:06.998265 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048616df98d8453bb82bc506c8fbf465)` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:07.000226 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:07.020910 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:23:07.046414 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048616df98d8453bb82bc506c8fbf465', 'request_id': 'req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:07.052459 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048616df98d8453bb82bc506c8fbf465', 'request_id': 'req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:07.057900 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048616df98d8453bb82bc506c8fbf465', 'request_id': 'req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8'} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:07.070301 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] The token cache is being invalidate because project 048616df98d8453bb82bc506c8fbf465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80748) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 21:23:07.070629 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:07.071314 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.notifications [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80748) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 21:23:07.073036 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1476/2911] 10.222.0.39 () {58 vars in 1232 bytes} [Wed Jul 13 21:23:06 2022] DELETE /identity/v3/projects/048616df98d8453bb82bc506c8fbf465 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:23:16.741527 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:23:17.067479 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:23:56.981170 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:23:56.982572 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:23:57.118284 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6d478-a5f5-40f3-b42e-a47194756797 admin admin] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:23:57.175866 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:23:57.176519 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:23:57.176831 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:23:57.177113 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:23:57.179041 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:23:57.190771 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:23:57.192338 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:23:57.202962 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1436/2912] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:23:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:24:07.197914 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=89 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:27:00.690561 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Authenticating user token {{(pid=80748) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:27:00.692043 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:27:00.771899 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde163d1-df9a-45d1-97c8-3e72cbbb4fb8 admin admin] Validating token access rules against request {{(pid=80748) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:27:00.775532 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df6ccbd929724922bdbb1150d543bacd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80748) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:27:00.776176 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] REQUEST_METHOD: `GET` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:27:00.776463 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] SCRIPT_NAME: `/identity` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:27:00.776745 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80748) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:27:00.778492 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.fernet_utils [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80748) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:27:00.788378 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:27:00.789806 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70ed8d-cc24-43be-af02-c1903461241c service neutron] RBAC: Authorization granted {{(pid=80748) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:27:00.798347 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: [pid: 80748|app: 0|req: 1477/2913] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:27:10.794134 dv-cinder-799871-27-iscsi devstack@keystone.service[80748]: DEBUG dbcounter [-] [80748] Writing DB stats keystone:SELECT=37 {{(pid=80748) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:28:59.980295 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] Authenticating user token {{(pid=80749) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:28:59.981669 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.fernet_utils [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80749) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:28:59.994454 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e8ffe-e08b-44a8-8f49-1fe6c668ef6e service placement] Validating token access rules against request {{(pid=80749) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:28:59.999530 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab409fb-c28d-4921-964d-66b4928be121 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab905149e23347d0aed37819978ac06f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0da876cd37f4d3a9e0d128652841317', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80749) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:29:00.000167 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab409fb-c28d-4921-964d-66b4928be121 service placement] REQUEST_METHOD: `GET` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:29:00.000451 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab409fb-c28d-4921-964d-66b4928be121 service placement] SCRIPT_NAME: `/identity` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:29:00.000729 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab409fb-c28d-4921-964d-66b4928be121 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80749) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:29:00.010017 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab409fb-c28d-4921-964d-66b4928be121 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:29:00.011600 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab409fb-c28d-4921-964d-66b4928be121 service placement] RBAC: Authorization granted {{(pid=80749) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:29:00.022057 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: [pid: 80749|app: 0|req: 1437/2914] 10.222.0.39 () {60 vars in 1365 bytes} [Wed Jul 13 21:28:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 21:29:10.017146 dv-cinder-799871-27-iscsi devstack@keystone.service[80749]: DEBUG dbcounter [-] [80749] Writing DB stats keystone:SELECT=6 {{(pid=80749) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}